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ABSTRACT 





Software process improvement initiatives are not unlike other process 
reengineering efforts. They are influenced by such dynamics as resistance to change, 
organizational structure, cultural barriers, and other issues. An effective plan for software 
process improvement implementation must address concepts of organizational change. In 
this thesis three perspectives on organizational change provide the frameworks for 
acaiteide the software process improvement efforts of four organizations. Based on the 
change theory and implementation strategies of four organizations best practices relative 
to preparing an organization for process improvement, implementing process 
sapreyenients afd sustaning the improvement effort are derived. A process improvement 
vey: archival material, personal interviews and site visits provide data on the process 
. improvement efforts of the Financial Systems Activity Kansas City. These data are 
anaieed to characterize the challenges to the organization's process improvement efforts. 
Recommendations for mitigating these challenges are provided. The recommendations 
include an explicit design for planned change, transition management teams, piloting, 

. integration of process improvement activities into the project cycle, and scanning - 


environment. 
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I. INTRODUCTION 

A. BACKGROUND 

In an age where one can barely keep up with the technological pace of computer _ 
hardware, the consistent production of quality software remains a mystery. While 
hardware development and production tends to be routine and predictable, poor quality, 
cost and schedule overruns, and project cancellations plague software development. 

That we have been in this "software crisis" for nearly three decades hints that 
there are some real ellie with software development. The requirement for quality 
software has increased constantly over the years. However, our ability to produce 
software has remained essentially stagnant (STSC, 1996). The state of software today 
does not reflect three decades of improvement, especially when compared to 
improvements in computing hardware over the same period. a 

There are many factors that contribute to the problems in software Revelopnient 
these include programmer skills, requirements analysis, user expectations, and project 
management. First, the software dlediuans industry has probably the widest variation 
in its level of sieiescional knowledge of any knowledge discipline (Osmundson, 1997). 
The disparity between a good programmer and one who is not can spell disaster for many 
development projects; and tight markets keep the best software engineers in short supply. 

Another factor is the adequacy of requirements’analysis and planning. The 
emphasis on producing software on schedule over producing quality software not only 
- detrimental effects on software quality but aie makes projects late. in their zeal 
_ to begin programming as soon as possible programmers take shortcuts in requirements 


analysis and specifications that are more costly in the later stages of a project. 








Moreover, many projects begin behind schedule because of poor cost and schedule 
estimates which often result in understaffing and unattainable deadlines. 

A third factor is customer expectations. Rapid developments in hardware fuel 
unrealistic user expectations for the pace of advancements in software (Lowry, 1991). 
These users also tend to change software expectations as a project progresses, often times 
shifting requirements through the well known phenomenon of "requirements creep" even 
| into the implementation phase. 

Finally, poor management and the absenice of process credibility are noted as 
primary contributors to software failures (STSC, 1996). Contrary to popular perceptions, 
software engineering professionals have indicated that software failures are more related 
to poor project management skills than they are to the technical issues of software 
development. | ) 

Software development as a discipline is not keeping pace with our growing 
reliance on computer systems. In the past, industry's attack on the software crisis has 
been focused mainly on develoonient of structured software methodologies and 
_ development of simpler programming languages and tools (Lowry, 1991). More 
recently, schvate process improvement is gaining momentum as the cure for software 
ills. Software process improvement activities describe those practices and procedures 
aimed at facilitating the consistent and predictable production of quality software (Paulk, 
1995). Improving software development processes may be the best way to mitigate the 
seoeaenaoned palace problems. As in other industries mature processes result in 


efficient production and consistent quality. 








B. SPI INFRASTRUCTURE 

The first step in improving software piocettes is to establish a supporting 
infrastructure. This infrastructure is the foundation for the activities, relationships, and 
structures necessary for the software process improvement effort. The form and content 
of a SPI infrastructure are as varied as the organizations that develop them. However, 
some baseline components of a SPI infrastructure are a software engineering process 
group (SEPG), a steering committee, process action teams, and a process improvement 
ante 

1. Software Engineering Process Group 

A software enginceend process group is an organization's focal point for process 
improvement activities. It is a group of specialists who facilitate the definition, 

‘maintenance, and improvement of the software process used by the organization (Paulk, 

* p. 63). This group takes the lead in implementing and sustaining process change. 
Effective SEPGs require a broad and diverse mix of skills including specialized software 
engineering principles, effective communications, negotiation, mediation, and 
_Salesmanship to name a few. The size of a SEPG is usually about 1-2% of the 
sig aieation's workforce. Typically the group is comprised of high achievers from 
various sections within the organization. 

2. Steering Committee 

Direction and oversight of the SPI effort is provided by a steering committee. The 
name of this enn may differ from organization to organization. The committee provides 
the vision and the resources to support the SPI effort. Membership in the steering 


committee usually includes the director, assistant director, senior managers, and the head 


of the SEPG. One of the most important functions of this committee is to communicate a 
clear vision of the organization's process improvement goals and support from the highest 
echelons of the organization. 

3. Process Action Teams 

Process action teams (sometimes called working groups) are the implementation 
arms of the SPI program. Members are skilled in specific disciplines of software 
engineering. Their tasks involve defining and implementing specific process — 
improvement isiiaiiiaas Their input is key in ensuring that a proposed change is realistic 
and in assessing the best means for transition. Participation in a process action team is 
often on a rotating or part-time basis. The size of the process action team varies with the 
organization. 

4. Related Software Groups 

| Related software groups are a collection of individuals (“oth managers and 


technical staff) representing a software engineering discipline that supports, but is not 


directly responsible, for performing software development and/or maintenance (Paulk, p. 


63). Two of the more important software related groups are the software configuration 
management group and the software quality agcunaiice group. In most organizations these 
groups are highly involved in the process improvement effort. 

5. Capability Maturity Model 

There are a couple of software process improvement models, including the 
fits national Standards Organization 9000 series (ISO 9000), the Software Process 


Improvement and Capability dEtermination (SPICE) and the Capability Maturity Model 





(CMM). This thesis examines organizations that use the CMM as the framework for its 
process improvement effort. 

The Capability Maturity Model is a framework that describes the key elements of 
an effective sna process. It is a tool for evaluating an organization's software 
development process. The model, based on industry best practices, covers all areas of 
software development including project planning, requirements analysis, software 
metrics, configuration sia ean and project management. It was designed to help 
developers select process improvement strategies by determining iit process maturity © 
and identifying the most critical issues to improving their software quality and process 
(Paulk, p.19). The model was created by the Software Engineering Institute (SED) a 
federally funded research and development center located at Carnegie-Mellon University 
. in Pittsburgh. The SEI was commissioned by the Department of Defense to help provide 
leadership in advancing the state of the practice of software engineering and to reduce 
variability and unpredictability in the software development process (Paulk, p.4). The 
first version of the CMM was released in 1991. 

The CMM consists of five maturity levels that describe the process éapability of 
an organization from adhoc processes to a ssaniions improvement environment. The 
CMM structure consists of maturity levels, key process areas, common features, and key 
practices. Figure 1.1 is a jae of the CMM structure. Each maturity level comprises a 
set of process goals that when satisfied stabilize an important component of a software 


process (Paulk, p.20). 








Maturity Levels 
Indicate 
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Process 
Achieve 
Organized by 
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Implementation or 
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Key Practices 
Describe 
Activities or 
Infrastructure 


Figure 1.1 The CMM structure (Paulk, p. 31) 


Paulk (1995) offers the following brief descriptions of the five maturity levels: 


1. Initial 


2. Repeatable 


3. Defined 


4. Managed 


The software process is characterized as ad hoc, and occasionally 
even chaotic. Few processes are defined, and success depends on 
individual efforts and heroics. 


Basic project management processes are established to track cost, 
schedule, and functionality. The necessary process discipline is in 
place to repeat earlier successes on projects with similar 
applications. 


The software process for both management and engineering 
activities is documented, standardized, and integrated into a 
standard software process for the organization. All projects use an 
approved, tailored version of the organization's standard software 
process for developing and maintaining software. 


Detailed measures of the software process and product quality are 
collected. Both the software process and products are 
quantitatively understood and controlled. 








5. Optimizing Continuous process improvement is enabled by quantitative 
feedback from the process and from piloting innovative ideas and 


technologies. 
Key process areas (KPAs) highlight the issues that must be resolved to achieve 
each maturity level. There are a total of 18 key process areas. These related activities 
when performed collectively achieve a set of goals considered important for enhancing 


process capability. Figure 1.2 is a diagram of the KPAs for each maturity level. 


O ptimizing 
Defect prevention 


Technology change management 
Process change management 


M anaged Pa 
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Software project planning 
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Initial 





Figure 1.2 Key process areas by maturity level (Paulk, p.33) 
Each key process area is described in terms of key practices. Key practices 


describe what is to be done to accomplish the goals of the KPA. These key practices 





describe the activities and infrastructure that contribute to the effective implementation 
and institutionalization of a KPA. 
The key practices for each KPA are organized by the following common features: 
e Commitment to perform typically involves establishing organizational policies 
and leadership to ensure the establishment and sustainment of a process. 
e Ability to Perform involves the resources, organizational structures, and training 
that must exist to implement a process. 
e Activities Performed describes the activities, roles, and ov necessary to 
implement a key process area. 
e Measurement and Analysis describes the basic measurement practices that are 
necessary to determine status related to the process. 
e Verifying Implementation describes the steps to ensure that the activities are 
performed in compliance with the process that has been established. 
These attributes are indicators of the effectiveness of the implementation and 
institutionalization of key processes. (Paulk, 1995) Refer to figure 1.1. 
6. Software Process Assessments and Capability Evaluations 
Software Process Assessments (SPAs) sia Software Capability Evaluations 
(SCEs), tools developed by SEI, are key parts of an SPI infrastructure. SPAs and SCEs 
are the measurement ake of the CMM. Process assessments are performed by software 
professionals to determine the state of an organization's current software process. In 
addition to identifying ackey software process related issues facing the organization 
they also generate support for process improvement efforts. A process assessment is a 


critical step in an internal process improvement program. (Paulk, 1995) 





Software Capability Evaluations are performed by software professionals to 
identify contractors qualified to ‘afer software work or monitor the state of the 
software process aed on an existing software effort. An organization's software process 
capability is one way to predict the likely outcome of the next software project it will 
undertake. Capability evaluations are performed by individuals external to the 
organization and are being used increasingly as a wieia in source sieceon by agencies 
with Department of Defense. Paulk, 1995) 

The CMM provides a general framework for process improvement. Howevei 
actual methods of implementation of CMM key practices are organizationally dependent, 
and implementing process improvement initiatives is challenging and time consuming. 
Much more needs to be learned about how the best software organizations are able to 
increase process maturity while others remain stagnant. The foundations of process | 
improvement are sound management and a commitment to eoaanious improvement. 
While these concepts seem obvious fey separate the few great organizations from the 
many mediocre and poor organizations. 

C. THE VALUE OF MATURITY. 
Itis ‘nuportain that leaders understand the value of a mature software process. 
Table 1.1 compares an immature software organization with a mature software 


organization. 





Immature Organizations Mature Organizations 






Software processes are improvised by practitioners during the Software processes are defined across the organization 
course of the project. 


Focus on improvement of processes through pilot studies, cost- 
| benefit analysis and other techniques. 
performance are usually achieved. 


Table 1.1 Immature versus mature software organizations 







Software process improvement is an expensive, complicated, and time-consuming 
undertaking. The SEI maintains a database of the maturity levels of organizations 
reporting CMM-based process improvement efforts. Figure 1.3 shows the organization 
maturity profile as of May 1998. The profile is compiled by SEI and characterizes the 
software process maturity of the software community. The profile is based on input from 
700 software organizations representing commercial industry, DoD contractors, military 
and other contractors worldwide. The profile indicalee the relatively immature state of 
software. These figures were compiled from a wate of organizations. At this time 
less than one percent of the assessed organizations are CMM Level 5 organizations. 
Process improvement issues and strategies from two CMM Level 5 organizations are 


reviewed in this study. 
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Figure 1.3 Organization Maturity Profile (Source SEI) 


D. BENEFITS OF CMM-BASED PROCESS IMPROVEMENT 

CMM-based process improvement is but one strategy that is available for an 
organization to improve its performance. Quantifying the return on investment of CMM- 
_ based process improvement is a contentious issue snnts industry leaders. The 
parameters for judging its shrecaveness are deny cia more qualitative than 
quantitative. However, in a technical report published by the SEI (SEI/CMM TR-13) data 
from a diverse group of 13 organizations (representing DoD contractors, military and 
commercial software organizations) were analyzed to assess the benefits of CMM-based 
process improvement. The report indicates significant gains in productivity, cycle time, 


and product quality from organizations using CMM-based process improvement. 





E. CHALLENGES OF SOFTWARE PROCESS IMPROVEMENT 

The challenges to process improvement are many. In the schedule driven world of 
software development, poorly planned process improvement efforts are quickly 
overwhelmed by delivery dates. Improved processes do not represent the tangible outputs 
(i.e., a completed project) that organizations are accustomed to. The connection between 
the strategic goals of the organization and process improvement efforts can be vague or 
nonexistent to employees. — improvement is not easy. It requires training on the 
part of all parties associated with the development process. Sluggish starts ‘i initial 
mistakes that are common in most software projects contribute to organizational 
resistance when they are encountered during the process improvement effort. Software 
developers of the "programming is art" mindset view attempts to manage the 
development process as a hindrance: Efforts at creating an environment of continuous 
process improvement must overcome the security that comes from familiar routines. 
F. CASE STUDIES | 

Because of the time, expense and complexity involved many organizations have 
abandoned their process improvement efforts (after incurring great costs). However, 
several case studies document well conceived and implemented process Sicaieiiad 
efforts. By examining the experiences of different types of software organizations and 
their approaches to software process improvement, a set of generalizable best practices 
may be derived. 

Software process improvement isitianives are not alike other process reengineering 

efforts. They are influenced by such dynamics as resistance to change, organizational 


structure, cultural barriers, and other issues. Software engineers must understand the need 
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to change, be convinced the new process will, indeed, improve performance, and be 
supported as they learn and implement it. (STSC, 1996) An effective plan for software 
process improvement implementation must address concepts of organizational change 
including sociotechnical change, transition management, cultural barriers, organizational | 
structure, and resistance to change. 
G. STUDY PURPOSE 

The purpose of the study is to develop ways to facilitate implementation of 
software process improvement initiatives. Initially change management theory will be 
examined to extract those themes relevant to successful implementation and sustainment 
of change. Then the software process improvement initiatives of a broad cross section of 
organizations are reviewed in the context of relevant change theory. By analyzing various 
‘implementation techniques.and ti associated results, best practices will be derived. | 

This research targets the Defense Finance and Accounting Service (DFAS), 
Financial Systems Agency, Kansas City (FSA-KC) which provides resources and support 
in performing a variety of software development services for the U. S. Marine Corps and 
other Department of Defense (DoD) organizations. FSA-KC is in the midst of a software _ 
process improvement effort that began in 1993. Information gathered from on-site 
analysis of FSA's current software process improvement efforts, interviews with FSA 
personnel, archival material, and a survey provide data for this thesis. 

The objective of this thesis is to apply software process improvement best 
sciciiias and change management theory to the Financial Services Agency (FSA) of the 
Defense Finance a Accounting Service (DFAS) in an attempt to facilitate their ongoing 


process improvement efforts. 
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H. RESEARCH QUESTIONS 
Primary: 
® How can the software improvement process be facilitated to increase efficiency? 
Secondary: 
e What is the software process improvement process? What problems does it face? 
e What are software process improvement best practices? 
e Howcan we apply change management theory to interpret software process 
improvement best practices? 
e What general guidelines can be developed to facilitate the improvement process? 
e What are the challenges to software process improvement efforts at FSA? 
e How can these challenges be mitigated using software process improvement best 
practices and change management concepts? 
I. METHODOLOGY 
The methodology used in this thesis research consists of the following steps: 
1. Conduct a literature search of journals, books, magazine articles, web 
resources, and other library information resources on change management 
concepts. 
2. Conduct a review of related change management theory. 
3. Conduct a eview of software process improvement case studies. 
4. Compile a list of software process improvement best practices based on 


common themes from successful cases. 
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>: Design and conduct semi-structured interviews for assessment of FSA-KC 
focusing on facts known to influence the effectiveness of improvement efforts 
based on data from case studies, consultant interviews, and literature reviews. 
6. Evaluate interview results and characterize FSA-KC environment. 
7. Prepare recommendations for implementation of software process 
improvement initiatives at FSA-KC. 
J. SIGNIFICANCE OF THE STUDY 
| This study provides recommendations to FSA-KC for successful implementation 
and maintenance of software process improvement initiatives. It serves as an example for 


other organizations seeking to implement software process improvement initiatives. 


15 


16 














Il. REVIEW OF RELATED LITERATURE 
A. INTRODUCTION 

Software process improvement initiatives are not unlike other process 
reengineering efforts. They are influenced by such dynamics as resistance to change, 
organizational structures, and cultural barriers to name a few. A review of related change 
management theory provides a context for examining SPI implementation issues within 
these constraints. Specifically, three theoretical perspectives on organizational change 
provide the frameworks for analysis in this study. 

Dalziel and Schoonover (1988) examine the problems of implementing 
organizational change and provide recommendations for mitigating hin problems. 
Pasmore (1988) approaches the problems of change implementation from the standpoint 
of optimization of the dindiiaiaile social ‘and technical systems within the constraints © 
of the environment it operates in. Finally, Bridges (1991) espouses the importance of 
managing transitions--the ending, the neutral zone, and the beginning--as the key to 
implementing and sustaining change. These three perspectives are reviewed below. This 
section concludes with a summary of the key themes. 

B. CHANGING WAYS 

In "Changing Ways", Dalziel and Schoonover (1988), suggest that effective 
change leaders deal with the tangible and hidden processes of change by answering three 
questions: 

e Have we got our organization ready for change? 


e Dowe have the right mix of skills on our team to make the change happen? | 


e Can we ensure that the implementation process will be successful? (Dalziel 
and Schoonover, p.15) 
1. Getting the Organization Ready for Change 
Dalziel and Schoonover (1988) offer the following as attributes of organizational 
readiness: 
e History of Change: The prior experience of the organization in accepting 
change. 
e Clarity of Expectations: The degree to which the expected results of change 
are shared across various levels of the organization. 
e Origin of the Idea or Problem: The degree to which those most affected by the 
change initiated the idea or problem the change solves. 
e Support of Top Management: The degree to which top management sponsors 
the change. - 
e Compatibility of Organizational Goals: The degree to which the proposed 
change corresponds to past and present organizational practices and plans. 
The first three dimensions--history of chan ge, clarity of expectations, and origin 
of the request or problem--focus on how to motivate people to embrace change. To 
stimulate a positive attitude toward change, change leaders let "success breed success", 
by scaling initial interventions to their organization's past performance (Dalziel and 
Schoonover, p.15). The most important thing is not to fail at the start. You should always 
ensure an initial success-even if you have to move more slowly or only start a small part 
of the change. In addition change leaders instill consistent expectations about the change 
and its ramifications throughout the organization. Finally, they make sure that change is 
framed to meet key problems of those who will have to live with the change first. 
(Dalziel and Schoonover, 1988) 


Support of top management is critical at the initial stages of planned change and 


important throughout the process. Upper-level alliances provide tangible support in the 


form of resources for various phases of the project and intangible support in the form of 
sponsorship and networking. Change managers managing successful endeavors generate 
support much more often than those producing less successful results. Dalziel and 
Schoonover (1988) suggest building a case for change that appeals to top-level concerns 
and seals a formal management review process involving key players in top- 
management as ways that change leaders can build top-management support. 

The best change leaders tailor the scope, scale, and type of change to fit existing 
patterns. This keeps the effort practical Dalziel and Schoonover, p17). A change that 
aligns with existing goals, practices, and priorities of the organization is more easily 
understood and adopted. Every leader who initiates a policy, program, or practice should | 
assess the compatibility of the change with the current organizational culture: 

_ @ Does the change fit with an overall business plan? 

e Do proposed changes make employees’ jobs harder or easier? 

_@ Is the change technically familiar to members of the organization? 

Dalziel and Schoonover (1988) suggest that change agents integrate the change 
into on-going procedures whenever possible. Another ead is to implement the , 
ere initially in the most accepting ial (1.e., a section with particularly 
knowledgeable and flexible workers). They also highlight the importance of 
communicating how the siaiies fits with sauanivational directions. 

2s Implementing the Change 

As the change effort evolves, the change leader must address a number of key 
issues to translate proposed actions into on-going organizational ciistiinn A set of five 


key processes support the process of successful implementations: 
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a. Clarifying Plans 

Clarifying plans is the process in which implementors define, document, 
and specify the change. Dalziel and Schoonover (1988) emphasize that successful change _ 
leaders make plans public and involve influential members of the end-user group to 
participate in formulating the plan. They conduct an on-going dialogue about each of the 
steps in their plan, with both the change agents and the end-users. 

b. Integrating New Practices 

Integrating new practices is the process in which an organization 
incorporates change into its operations. Change managers should gradually integrate 
change efforts into an organization, gearing the rate of change to the organizational 
context rather than cramming it into a prefixed timeline. Limiting the rate of change until 
acceptance and understanding is clear produces more effective long-term results. It is also 
important to limit the amount of change introduced at one time. 

c. Providing Education 

Educaton provides programs in which end users learn about and use new 
approaches and procedures. A good training program is designed from the user's 
perspective. It incorporates their knowledge and experience, and is sonaantly evaiunted 
in the context of its affect on work practices and user attitudes. 

d. Fostering Ownership 

Fostering ownership is the process through which end users come to 
identify new processes and procedures as their own, rather than regarding them as 
changes imposed upon them. Ownership is fostered when the primary benefits of the — 


change are apparent to the end-users and when the end-users are involved in the plans, 
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decisions and outcomes of the effort. When users understand and participate, they 
become committed to current and future strategies. 

e. Feedback 

The feedback process is accomplished by first documenting and 
describing the expected outcomes of the change, and then using input from those affected 
by the change to judge the effectiveness of the change during and immediately after 
implementation. Effective sind managers perform significant up-front sc fomaetoll | 
gathering andieoieane exploring options throughout the change effort. Aeseing 
employee and management attitudes and practices is key to successful change 
implementation. The broad perspective gained from multiple inputs. that include various 
business functions and customers often generate ideas for more practical effective change 
efforts. (Dalziel and Schoonover, 1988) 
C. THE SOCIOTECHNICAL SYSTEMS PERSPECTIVE 

Pasmore (1988) asserts that an innate effectiveness is a function of the 

people (the social system), the tools and techniques shay use (the technical systems) and 
the customers (the environment). By optimizing the social and technical elements of 
sreaioatonal iden within the context of the environment, organizations may scien | 
peak performance and efficiency. Conversely, failures in organizational undertakings can 
be traced to overlooking the significance of one or more of these elements. While it may 
be argued that there are other factors that contribute to organizational effectiveness, it 
sin clear that at a minimum these three watiet be posers Pasmore asserts that every 


organization is a sociotechnical system. 
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1. The Environment 

_ Every organization is ultimately dependent on the environment (in the form of 
customers, competitors, sources of resources, and other things) for survival. The 
environment imposes constraints and opportunities that influence the goals, processes, 
and outcomes of organizational systems. Internal measures of success that are not 
consistent with environmental constraints and opportunities are inaccurate predictors of 
an organizations future success. The external environment is the ultimate judge of system 
siietboaiias The sociotechnical systems perspective asserts that whatever decisions are 
made about organizational design should meet the demands of the external environment 
as well as the internal social aaa technical systems. (Pasmore, 1998) 

As a basic tenet, organizational designs should "fit" with the environment (Kotter, 
1978) but rapid and unpredictable changes in the environment make achieving the proper 
fit difficult. To achieve and maintain an optimal fit between the organization and the 
environment, the organization must also change continuously. Therefore managers 
should strive to create a learning capacity within the organization, to facilitate continuous 
_ improvement of organizational practices and systems. (Pasmore, 1988) 

Sannin asserts that the greater the level of experimentation in organization 
design, the greater the likelihood that learning will occur and lead to more effective 
future adaptations to the environment. However, he realizes that opportunities for 
managers to experiment with different techniques that saints their learning is difficult 
in the face of peste to settle upon a workable solution that can deal with immediate 


environmental demands. (Pasmore, 1988) 
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The environment can be seen as a provocation or a source of inspiration 
(Pasmore, p21). The more the environment is viewed as a source of provocation, the 
more adaptation will focus on solving immediate problems versus innovations in 
organizational design. Companies that are successful in turbulent environments do more 
than react to competition; they take steps to transform the environment itself to make it 
more conducive to their well being. (Pasmore, 1988) 

Given the impact of the environment on organizational success, it is clear that 
sociotechnical systems design must begin with an appreciation of organization- 
environment relationships. Pasmore (1988) details two methods for seannine the 
environment; open systems planning and the search conference. The methods are very 
similar. Both methods assess the organization-environment fit, within the context of a 
particular change initiative. Table 2.1 illustrates thé general approach to the two scanning . 


methods. 


Open Systems Planning | Search Conference 
List the important stakeholders in the environment and 
expectations they hold regarding both how the organization will . 
erate and what it will achieve. | ae of 
Create a realistic future scenario which depicts what would 
happen if it continued on it's current course. 
Compare the realistic and idealistic scenarios to identify design 
constraints and opportunities. 
Begin operational planning to achieve the desired future system. 
future scenario. 


Table 2.1 Environmental Scanning Methods 
























_ These techniques for scanning the environment can be carried out at different 
levels of the organization. Environmental awareness is enhanced by regularly involving 


employees at all levels in scanning the environment. Regardless of the mechanisms 
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employed there is greater potential for generating workable action plans when key 
external stakeholders are involved in scanning the environment. (Pasmore, 1988) 

2. The Social System 

Organizational success depends on social system dynamics. Therefore, the social 
system should receive as much attention during sociotechnical systems design as the 
technology and the environment (Pasmore, 1988). The social system of an organization 1s 
comprised of the people who pee in the organization and all that is human about their 
presence. This includes individual attitudes and beliefs, the implicit scucheleceal 
contracts between employees and employers, reactions to work arrangements and 
company policies, and a myriad of other human dimensions. Pasmore (1988) points out 
that analyzing the organization as a social system is difficult, but social systems are the 
only parts of the organization that ari conceive and implement improvements in 
organizational processes. 

While encouraging managers to ative for joint optimization of social and 
technical systems, Pasmore (1988) concedes that actual optimization of social and 
technical systems is extremely challenging considering the complexity and rapidly 
evolving amie of social systems. However, he points out the following sesitonitl 
aspects of social systems: 

e The better the fit between the organization's culture and its external 

environment, the more effective the organization will be. 

e The greater the disparity between seeanliniticing! design features and the 

unique characteristics of organizational members, the less successful a design 


will be. 


24 





e The greater the involvement of employees in the design process: the re 

flexible the resulting organizational design is likely to be. 

e The greater the involvement of employees in the design process, the ears 

the understanding of how behaviors are linked to desired rewards. 

Pasmore (1988) asserts that at the macro level, social system dynamics can be 
understood both in terms of the culture of the organization and its structure, which 
includes the designation of departmental boundaries, reward systems, supervisory and 
control systems, job design stinciples, performance expectations, employee involvement 
opportunities, and the nature of contracts between labor and management as well as 
psychological contracts between the organization and its members. The challenge for the 
sociotechnical systems designers is to first understand the constraints that social 
dynamics within an organization place on design possibilities, and second, to recognize - 
that bold design changes can produce both desirable and undesirable effects on behavior. 
(Pasmore, 1988) 

The social system is the source of all innovation and adaptation. Because 
_innovation and adaptation require human commitment, we need to understand how the | 
design of seeanizalions influences the willingness of people to help organizations 
succeed. High performance requires commitment; building commitment requires treating 
people like adults and engaging them fully in shaping their future. (Pasmore, 1988) 

3. The Technical System 

Pasmore defines the techincal system of an organization as the tools, techniques, 

devices, artifacts, methods, configurations, procedures and knowledge used by 


organizational members to acquire inputs, transform inputs into outputs and provide 
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outputs or services to clients or customers. The primary function of technology is to 
enhance the amount of work an individual can accomplish and the reliability of 
individual performance. (Pasmore, 1988) 
The nature of the technical system used by an organization influences the 

apparent level of commitment and motivation demonstrated by individuals. The interplay 
between the roles defined by technology and one's self image gives rise to psychological 
contracts between the individual and organization which define the level of effort and 
commitment one will demonstrate in pursuing organizational goals. Technologies which 
prescribe narrow roles for individuals, produce psychological contracts which preclude 
learning or change, and are reinforced by structural arrangements which interfere with 
cooperative problem solving may result in total immobilization. Under such 

_ circumstances changes of almost any kind are threatening, since they may undercut the 
personal or political security associated with traditional arrangements (Pasmore p.62). 
The design of jobs will be more stimulating when the technology is designed to provide 
more direct and immediate feedback and when the technology leaves a significant degree 
of relevant decision making to employees. 

4. Sociotechnical Systems Redesign 
The sociotechnical systems redesign model (Figure 2.1) is targeted at 


implementing changes to existing systems. This model for redesign is consistent with 
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Figure 2.1 Sociotechnical Systems Change Model (Adapted from Pasmore p.111) 
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those concepts put forth in the sociotechnical systems perspective. Pasmore offers 
the following cautions on the use of the sociotechnical systems redesign model: 

It is not a quick-fix, limited scope, cosmetic process which is tightly controlled by 

management alone. If followed as outlined here, the mode] may take anywhere 

from six months to three years to complete; it will probably demand full-time 
attention by several organizational members, require a substantial outlay of 
resources, and call into question every aspect of the organization's design and 
operating practices; moreover, it 1s almost certain to disrupt organizational 
performance for a period of time. But in the end, it is more likely to produce 
changes that are desired--and often demanded--in a competitive environment 

(Pasmore, p. 110). 

Each of the items in the sociotechnical systems redesign process is briefly 
described below: 

Step 1: Define the scope of the system to be redesigned. 

The first step in the sociotechnical systems change process is intended to clarify 
the boundaries of the organizational unit to be redesigned. In addition, it includes the 
activities associated with the entry, scouting and contracting phases in typical 
organization development.efforts. These include defining the need for change, 
determining the potential for success, agreeing on a change mode, defining rough time 
and cost parameters, clarifying =xpectations among parties, forming a steering committee - 
to oversee the effort, and agree1iiz to a public contract which outlines the work to be 
done. 

Step 2: Determine environmental demands. 

The second step in the change process involves identifying important 


constituencies in the external environment who might impact the nature and scope of 


changes that will occur in the organization. Based on the perceived demands of 
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competitors, shareholders, corporate management and others, areas of opportunity as well — 
as constraints on the edesiea are identified. 

Step 3: Create the vision statement and charter. 

Based on the environmental demands identified in Step 2, top level decision 
makers draft a preliminary statement outlining their vision of the ideal organization they 
_ hope to create through the change process. In addition, the statement outlines the 
perceived constraints in such areas a policies, reward systems and labor agreements and 
sets clear goals for performance after the redesign. 

Step 4: Educate organizational members. 

Although designated as a distinct step in the change process, the education of 
organizational members which begins in Step 4 continues throughout the reniainde: of 
the change process and beyond. In addition to. education about the sociotechnical systems 
perspective and change process, organizational members should also. begin training that 
will prepare them to assume their new roles in the redesigned system. 

Step 5: Create the change structure. 

In step 5, a representative design team is formed which will conduct the actual 
sociotechnical systems analysis and formulate proposals for adi in the epeettne 
relationship between the design team and the top level decision makers on the steering 
committee is clarified. 

Step 6: Conduct the sociotechnical systems analysis. 

Step 6 actually consists of three distinct but interdependent analyses of the social 


system, technical system, and environment of the organization as detailed in the previous — 
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sections. A clear understanding of how the organization currently operates points toward 
areas for improvement in the future. 
‘Step 7: Formulate redesign proposals. 

Ideas for redesigning the organization flow from the analyses conducted in Step 6 
as well as the vision put forth by the steering committee and revised by organizational 
members. All proposals are reviewed by organizational members and checked against 
sociotechnical systems principles. 

Step 8: Implement recommended changes. 

In Step 8 a plan is devised for implementing the changes which pane the review in 
Step 7. Responsibility for seeing that the changes occur is designated to individuals with 
the power to make them happen. A review system is created to monitor implementation 
success.- 

Step 9: Evaluate changes and redesign . necessary. 

Since the process of sociotechnical systems change is complex, it is necessary to 
evaluate the changes that are made to ascertain whether they are producing their intended 
effects 
D. MANAGING TRANSITIONS 

The leader is no more or less than a manager of transitions (Pasmore, p. 148). 
Bridges (1991) distinguishes change management theory as focusing on a desired 
outcome that the change will produce while transition management focuses on the 
psychological process people will go through to come to enn with the new saation 
Bridges terms the three phases of transition as the ending, the neutral zone, and the new 


beginning respectively. This is not to imply that the phases are distinct and sequential, but 
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more to isolate the dynamics of transition into its simplest forms. As depicted by the 
overlapping phases in Figure 2.2 the organization will be in more than one phase at a 
time. The movement through transition is marked by a change in the dominance of one 


phase as it gives way to the next. (Bridges, 1991) 





Figure 2.2 The Three Phases of Transition (Adapted from Bridges p. 70) 


_ Transition management recognizes the fact that one of the major challenges of 
implementing and sustaining change is getting people to stop doing the things that they 
have grown accustomed to doing. It also recognizes the significance of that time between 
the ending of established practices and the beginning of new practices. This is a time of 
increased organizational vulnerability, but can also be a anie of increased creativity and 
ingenuity. Unless transition occurs, change will not work. (Bridges, 1991) 

1. The Ending | 

While the first task of change management is to understand the destination and 
how to get there, the first task of transition management is to convince people to leave 
sone (Bridges, p.32). The starting point for transition is not the outcome but the ending 
that you will have to make to leave the old Situation behind. If things change at least 


some employees and managers are going to have to let something go. So beginnings 
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depend on endings. Bridges (1991) cites the connection between resistance to change and — 
the losses associated with endings. It isn't the changes themselves that people in these 
cases resist. It's the losses and endings that they experience and the transition that they 
are resisting. The failure to identify and be ready for the endings and losses that change 
produces is the largest single problem that organizations in transition encounter. 

Bridges (1991) recommends providing frequent and continuous communication as 
a solution for managing the impact of the ending on people. Change agents should 
describe the change in as much detail as possible. They should present the innovations as 
developments that build on the past and help realize its potential. 

2. The Neutral Zone 

The neutral zone is the no-mans land between the old reality and the new. It isa 
time when the old way is gone and the new one doesn't feel comfortable yet. The neutral 
zone is usually a time of confusion and frustration, but if managed properly can be a time 
of increased creativity and innovation. The more drastic the change the longer the time 
spent in the neutral zone and the more important it is to effectively manage this time. 
(Bridges, 1991) 

To minimize the confusion and frustration in the neutral zone senior management 
must protect their people from unrelated or unnecessary changes as much as possible. If 
this 1s not possible, management should try to cluster and relate new changes to the 
primary on-going changes. People can deal with a lot of change if it is coherent and part 
of a larger whole. But unrelated and unexpected changes, even if they are small can be 


the proverbial straw that breaks the camel's back. (Bridges 1991) 
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One of the persistent problems during transition is for decision makers and those 
implementing decisions to be clear on precisely what impact the decisions and actions 
will have. A transition monitoring team (TMT) can provide feedback and facilitate 
communication between planners and implementors. The TMT's purpose includes: 

e Demonstrating organizational concern for the impact of transition on the 

people. 

e Reviewing plans or communications before they are announced. — 

e Providing ssaayiacceds to the organization's grapevine. Countering rumors 

and correcting misinformation. (Bridges, p.42) 

In the neutral zone restraints on innovation are the weakest. Only when the old 

way of seeing things disappears are habit patterns Boker: ail anew way will emerge. 
Innovation will take place automatically in the neutral zone if people are protected from 

- further change, ceteane and encouraged to find new ways to do things. (Bridges, 1991) 
3. The New Beginning 
A new beginning can only take place after a well managed ending and neutral 
_ zone. Starts involve new situations, a beginning involves the new understandings, values, 
ai attitudes that are required for successful transition. To make a new beginning people 
need The Four Ps: the purpose, a picture, the plan, and a part to play. (Bridges, 1991) 

a. The Purpose 

Successful new beginnings are based on a cea purpose that aligns with 
the as eeaal characietetes of the sronatoation and the eternal forces that influence the 


organization. You need to explain the purpose behind the new beginning clearly. You 
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may discover that people do not have a realistic idea of where the organization really 
stood and what its problems were (Bridges, p. 53). 

b. A Picture 

Most of the pain of the neutral zone comes from the fact that it is a time 
without a viable organizational picture. People in the organization need to be able to 
visualize the change. They need something they can see, at least in their imaginations. 
They need a picture of how the outcome will look and they need to be able to imagine 
how it will feel to be a participant in it. (Bridges, 1991) 

c. The Plan 

A transition management plan differs from a change management plan in 
that it starts with where the people are and works forward through the process of ending, 
‘getting through the neutral zone, and finally the new beginning. The transition plan oui 2 
with where people are and spells out the details of the route form the current state to the 
future state. Plans are immensely reassuring to most people, not just because they contain 
information but because they exist. (Bridges, 1991) 

d. A Part to Play 

Even the best-laid plans leave a troubling doubt in the minds of some 
people. Until people are given a part to play they will feel left out and find it difficult to 
make a new beginning. Bridges indicates that people should have two parts to play. First 
they need to see their role and their relationship to others in the new scheme of things. 
They should also have a role in lillies effectively with the transition process itself. The 
easiest way to do this is to be sure that everyone has a place on a planning task force, 


climate survey group, problem-solving circle or transition monitoring team. If this is not’ 
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possible, set up formal input systems for such groupings so that each person has at least 
an indirect part to play in the transition management process. Commitment to the new 
beginning is enhanced by involving as many people as possible in the plan. Everyone 
who plays a part is implicitly implicated in the outcome. (Bridges, 1991) 
4. Reinforcing the New Beginning 
Bridges indicates that beginnings that are not reinforced will revert to chaos when 
confronted by the sGuiatng miei of changes that are sure to come along. To reinforce 


the new beginning, senior management must ensure that they are sending consistent 


‘messages about the importance and the priority of the change. If you tell people they need 


to do five new things but don't remove anything from their list of tasks, you are sending 
conflicting messages (Bridges p.61). Conflicting messages are confusing in their own 
right and also provide people with re to argue that the new beginning isn't for real. 
(Bridges, 1991) 

It is common and disastrous to tell people to act and react in new ways and then o 
reward them for the old actions and reactions. People have to feel that they are better off 
for having changed their attitudes and behavior, and if they don't you'd better look at your 
reward oan Bridges, 1991) | | 
E. SUMMARY 

A number of significant issues related to change implementation were presented 
in the work of Bridges (1991), Dalziel and Schoonover (1988), and Pasmore (1988). 
er their appieaetes differ, there is orks overlap in ae coverage of certain 
concepts. The overarching themes of the literature addressed assessing an organization's 


readiness for change, change implementation strategies, and sustaining change. 
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1. Organizational Preparation 

The literature indicates the importance of assessing the organization's preparation 
for the planned change. Internal dimensions of organizational preparation include the 
culture and attitudes of the workforce, the workforce's recognition of the problem that the 
change addresses, and management support. These internal dimensions of organizational 
preparation can be managed through communication, selling the problem, and the use of 
information-gathering techniques to assess workforce concerns. External dimensions of 
seatiabeanl preparation reflect the adequacy of open systems planning or a search of 
environmental factors such as customers, competitors, market conditions, and legislation. 
Managers must have a clear uadersanaiie of the impact of environmental factors “ the 
implementation of the planned change. Scanning the environment is a technique for 
assessing and working within the constraints of the organization's external environment. 

2. Process Improvement Strategy 

The literature review highlighted the significant attributes of a change plan. The 
plan should be detailed and targeted at all levels of the organization and encourage 
involvement by as many people in the organization as possible. The plan should ensure 
quick sees initially and align with the business goals of the organization. In 
addition, changes should be piloted on a representative project where possible. 

3. Sustaining Change 

The literature emphasizes the importance of establishing a culture where "change 
is the norm” in an Sreanieaion for sustaining change. Techniques for establishing such a 
culture include encouraging innovation, not punishing failure, ongoing training and 


development, and reward systems that are consistent with a learning culture. 
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In the section that follows data from software process improvement case studies, 
technical reports, and other papers will be examined within the context of the 
aforementioned issues. 

F. SOFTWARE PROCESS IMPROVEMENT CASES 
1. Introduction 
In this section the software process improvement efforts of ut organizations are 


reviewed: 


Boeing Defense and Space Group (D&SG), Space Transportation Systems 
(STS) | 
e Hughes Aircraft Software Engineering Division (SED) 
e The Software Engineering Laboratory at NASA Goddard Space Flight Center 
e Raytheon Electronic Systems Organization . 7 | 
These organizations were selected for review because of their outstanding 
reputation for software process improvement and because of the rich data available on 
their efforts. Hughes (1997), NASA/GSFC (1994), and Raytheon (1995) are winners of 
| the IEEE Software Process Achievement Award, and Boeing STS is one of only four 
CMM level 5 comeing listed on the current organization maturity profile maintained 
by SEL It is not the intent of this thesis to enumerate all process improvement activities 
of these organizations; rather the intent is to derive those best practices that are likely to 
contribute to the greater knowledge of effective software process improvement activities. 
2. Boeing Space encanta Systems | 
In July, 1996 the Boeing Defense and Space Group (D&SG), Space . 


Transportation Systems (STS) organization achieved a level 5 rating on the SEI’s CMM. 
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A series of articles relating Boeing's experiences in achieving CMM level 5 provide 
insight into the process. 

George Yamamura, Software Engineering Process Manager for Boeing STS, 
attributes Boeing's success in process improvement to an approach that is responsive to 
business goals and an emphasis on the human issues of process improvement 
(Yamamura, 1998). In a presentation given at the March, 1998 SEPG conference, 
Yamamura recommended conducting surveys to assess worker's perspectives on issues of 
importance, levels of satisfaction, and areas of improvement. He sunsests using employee 
inputs in conjunction with an analysis of the situation to develop a change strategy. 

Boeing's Management Goal Framework is a practical example of clarifying plans 
as prescribed by Dalziel and Schoonover (1988). It provides a roadmap for the business 
_ case. It starts with the organization's business goals and breaks them down in successive 
levels until they are translated to process activities at the project level. 

Boeing's four’step formula for success, RSST: Right thing, Small steps, Simple, 
and Timing can be summarized as understanding the problem within the context of the 
environment, looking for simple, appropriate solutions, ad implementing in small steps 
to gain quick successes. | 

Boeing's mechanism for evaluating candidate process changes (shown in figure 
2.2) is of particular si eniticance to this thesis. It provides a framework for satisfying 
some of the key requirements for successful change implementation, including gaining 
management support, piloting the change, monitoring, and feedback. (Kness & Satake, 


1997) 
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Figure 2.3 Boeings Process Improvement Process 
(Adapted from Kness & Satake, 1997) 

After an improvement plan is made and initial management support is gained, the 
potential process improvement initiative is piloted on a representative software 
development project. Once the benefits of the piloted improvement are evaluated, a gate 
‘review is conducted with management to determine if more widespread fnidldntatisiiein 
iS warranted. After which, the improvement may be implemented on other projects. The 
implementation of the improvement is then closely monitored. A workin g group 1s 
established to address issues of refinement of the improvement and training and to 
develop lessons leamed. Boeing indicates that this approach helps prevent waste of 
resources in process changes that do not aeeve the desired results. (Kness & Satake, 
1997) 

Boeing's approach to SPI is consistent with théoretical concepts of preparing the 
organization for change (via information gathering and an awareness of the environment) 
aad sound change implementation strategies (via Miseihan with business seals and 


clearly defined plans). 
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3. Hughes Aircraft Software Engineering Division 

Hughes Software Engineering Division (SED), formed in 1978, primarily works — 
on US Defense Department contracts. It employs about 500 professionals. Of these, 41 
percent have 10 to 20 years experience in software and 12 percent have 20 or more years 
experience. In 1990, SED was assessed at CMM level 3 by the Software Engineering 
Institute. The two-year program of improvements cost $400,000. Hu ghes found that the 
investment improved working conditions, employee morale, and the performance of the 
SED as measured in project schedule and cost. Hughes estimates the resulting annual 
savings to be about $2 million. (Humprey, Snyder, & Willis, 1991) The factors that 
contributed to Hughes success include top management support, alignment with business 
goals, employee involvement, and a comprehensive technology transfer function. 

Hughes realized that implementation of the SPI action plan as ori ginally drawn up = 
would require additional funding. Consequently support and involvement of top 
management (who reviewed the action plan prior to approving additional funding) was an 
early requirement. | 

The action plan listed the goals of the SPI effort and put the action plan in the 
context of a Ground Systems Group Organizational Improvement Strategy. The action 
plan also included an estimation of the labor required for implementation. Additionally 
quality indicators (i.e., error and dies counts) were identified, collected and presented to 
senior management in monthly briefs on the health of each project. (Humphrey et. al., 
1991) | 

Hughes credits the creation of a Head of Technology Transfer function as the 


most profound action in the entire improvement process. 
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Among other things, the head of technology transfer coordinated self- 

assessments, developed a questionnaire glossary, became the local expert in the 

SEI maturity questionnaire, became a member of the Software Productivity 

Consortium's technology-transfer advisory group, developed an SPC technology 

transfer plan, briefed senior management on the state of process maturity, 

maintained a database of technology used on each project and an awareness of 
what technology each project needed, facilitated technology transfer among 
projects, and a special-interest group on process improvement, supported the 
corporate wide technology transfer program, served on the practices and 
procedures change review board, the training policy committee, and the 

technology steering committee (Humprey et al., p.18). 

While it is perhaps more impressive that one person could effectively handle so 
many tasks, the detailed breakdown of these tasks reflects the meticulous.nature of 
Hughes SPI plan. 

Hughes’ SPI implementation strategy is illustrative of a number of the theoretical 
concepts for successfully implementing change. Their early involvement of top 
management and emphasis on employee feedback are practices consistent with Dalziel 
and Schoonover's (1988) principles for preparing an organization for change. The many 
tasks of the Head of Technology Transfer seem to be aimed at optimizing the fit between 
new technologies and the people as Pasmore (1988) advocates. These tasks also provide 
mechanisms for gauging and guiding the impact of technical changes on the organization, 
consistent with Bridges (1991) guidance on managing the transition. This SPI strategy 
allowed Hughes to progress from CMM level 2 in 1987 to a strong level 3 in 1990. In 
1997 Hughes was the recipient of the Software Process Achievement Award. 

4. The Software Engineering. Laboratory at NASA Goddard Space Flight 
Center (GSFC) 
A partnership between NASA/GSFC, the University of Maryland, and Computer 


Sciences Corporation forms the Software Engineering Laboratory which was created in 
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1976 for the purpose of understanding and improving the overall software process and 
products that were being created within Flight Dynamics Division (FDD) of NASA SEL. 
SELs continual experimentation with software process has yielded an extensive set of 
empirical studies that has guided the evolution of standards, policies, management 
practices, technologies and training within the organization. Trends have been observed 
from as far back as 17 years. (McGarry, Pajerski, Page, & Waligora, 1994) NASA SEL is 
the first organization to win the IEEE Computer Society Software Process Achievement 
Award. This organization could represent the pinnacle of SPI hinleaneiae Factors 
that contribute to their success include: a bottom-up approach to SPI that emphasizes 
local goals, characteristics, and product attributes, alignment with business objectives, 
and a learning culture. 

Incorporating the key eoneent of change guided by development project 
experiences (McGarry et. al., 1994), the SEL defined a standard paradigm to illustrate its 
concept of software process/product leuaciiaeiek This paradigm is a three-phase wal 
(figure 2.4) which includes the following steps: 

1. Understanding: Improve insight into the software process and its products 
by characterizing the production environment, includin 9g types of dieiiae 
developed, problems defined, process characteristics, and product 
characteristics. 
Zs Assessing: Measure the impact of available technologies and process 
change on the products science Deenatie which technologies are 


beneficial and appropriate to the particular environment and, more 
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importantly, how the technologies (or processes) must be refined to best 
match the process with the environment. 

3. Packaging: After identifying process improvements, package the 
technology for application in the production organization. This includes 
the development and enhancement of standards, training, and development 


policies. (McGarry et. al., 1994) 
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Figure 2.4 NASA SEL 3 Phase Model (Adapted from McGarry et. al., 1994) 


Change is driven by product and not merely process alone (McGarry et. al., 
p.40). SEL's philosophy that if a changed process has no positive impact on the product 
generated then there is no justification for making the change, closely links SPI efforts 
with local business objectives. The SEL process improvement strategy cinphacizes a 


baseline understanding of the local software process, products, and goals. SEL 
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distinguishes their "bottom-up" approach from the typical top-down approach in the 
following way. (McGarry et. al., 1994) 

‘The top-down approach is based on the assumption that there are generalized, | 
universal practices that are required and effective for all software development, 
and that without these practices, an organization's process is deficient. This 
concept does not take into account the performance issues, problems, and unique 
software characteristics of the local organization. The goals and characteristics of 
the local organization are not the driving elements of change. 

In contrast the underlying principle of the SEL approach is that "not all 
software is the same.” Its basic assumption is that each development organization 
is unique in some (or many) aspects. Because of that, each organization must first — 
completely understand its local software business and must identify its goals 
before selecting changes meant to improve software processes. If, based on that 
understanding, change seems called for, then each change introduced is guided by 
"experience"—not by a generalized set of practices. (McGarry et. al 1994) 
Process change has been infused as a standard business practice at NASA SEL. 

Each production project in the Flight Dynamics Division is considered an opportunity for 

the SEL‘to expand its knowledge of process understanding and improvement. SEL 

analysts identify software process modifications that they hypothesize are likely to 

improve the resultant product and then design an experiment to test the hypothesis. The 

SEL conducts three general types of analysis, all of which are active continually in the 

. environment. They include: 

e Pilot studies of specific techniques and technologies on a project or set of 
projects. 

e Studies of completed projects for development and refinement of local process 
and product models. 


e Trend analysis of completed projects to track the impact of specific process 


changes on the environment as a whole. (McGarry et. al. 1994) 
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SELs emphasis on product improvement over process improvement ensures that 
changes are compatible with existing organizational practices and plans as prescribed by 
Daiziel and Schoonover (1988). Their view of every project as a learning opportunity 1S 
consistent with Pasmore's (1988) assertion that the greater the level of experimentation in 
organizational plans the greater the likelihood that learning will occur and lead to more 
effective adaptations to the environment. 

5. Raytheon Electronic Systems 

Raytheon Company is an international company that operates in four business 
areas: commercial and defense secunies: engineering and construction, business 
aviation, and major apliances Raytheon Electronic Systems focuses on commercial and 
defense electronics. RES is a recently consolidated organizational entity that emerged 

from the restructuring of Raytheon’s defense business in 1995. (Haley, Ireland, 
' Wojtaszek, Nash, & Dion, 1995) | 

In the fall of 1987 prompted by the lack of success in delivering software projects 
on schedule and within budget, the Equipment Division of Raytheon (now part of RES) 

_ initiated a SPI effort known as the Software Engineering Initiative. In 1995 the 
Bquipnient Division was the recipient of the Software Process Achievement Award. 
Factors that contributed to Raytheon’s success include: an emphasis on quantitative 
measures of process improvement, a well defined implementation plan, and continuous 
feedback. 

A sontinuiods concern that the $1 million annual expenditure of discretionary 
funds on SPI was not really achieving a return sufficient to justify not eoenaits the 


money on something else fueled Raytheon’s drive for quantitative measurements of 
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process improvements. Raytheon measured the return on investment on SPI based on: the 
amount of rework, the predictability of software development effort, and software 
product quality. (Haley, et. al., 1995) 

Raytheon’s process improvement strategy consists of the organization's standard 
software engineering process which is defined by an underlying RES software policy. 
This policy describes the set of common software engineering practices: the “whats” of 
developing software, detailed procedures describing the how of critical aspects of 
software iesaieaianl along with the tools and the training needed to make the 
developers productive (Haley et. al. p.3). 

Raytheon’s SPI strategy (Figure 2.5) includes feedback mechanisms, is 
responsive to the environment and provides guidance at all levels of the organization. The 
effectiveness of existing procedures are measured and analyzed, with the results of the 


analysis continuously fed back into the process improvement effort (Haley et. al p.19). 
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Figure 2.5 Raytheon's SPI Strategy (Adapted from Haley et. al., 1995) 

Raytheon's "Blue Books” are a three-tiered set of documents which define 
Ravihiesats en at the highest level (Software Engineering Policy), the iideaniadints 
level (Software Engineering Standards), and the lowest level (Detailed Procedures and 
Guidelines). Defining the plan in this way has enhanced “buy-in” to Raytheon’s 
improvement process at all levels of the organization. (Haley et. al., 1995) 

RES io extended its practice of Key poem Reviews of he major projects in each 


business area (chaired by the General Manager) to include the review of process 
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improvement accomplishments by each of the functional areas within RES. This elevates 
process improvement activities to the level of project activities and keeps levels linked. 
Any particular project uses the organization’s process, consciously tailored to its 
particular needs and constraints along with its own project software development plan, 
the key document binding the project to the process. The plan is typically constrained by 
the contract, the statement of work, and the requirements of the system as specified by 
the customer or developed early on during the system definition. As the project software 
engineering éeeurs and the specific process is applied, two types of feedback take place. 
At the project level, the software development plan is refined to reflect lessons learned in 
early phases of the seeiconeit and at the organizational level, these lessons learned 
will have an impact on the process improvement activities and eventually lead to the 
creation of generic solutions to be added tothe organization's standards. In the meantime, 
' the process improvement activities being conducted by the initiative are benefiting from 
the real time application of these solutions on projects. The project feedback along with 
outside influences such as technology drivers, the marketplace, corporate business 
_ decisions, customer initiatives, all have an impact on the direction in which process 
need will occur. (Haley et. al., 1995) 
| Raytheon’s unique approach to training also enhances organizational “buy-in”. 
All courses are given during working hours, which promotes the feeling of company 
support and enhances morale. A detailed feedback questionnaire is completed by the 
student at the iacilelils of the course. Further, during the transition phase, process 
improvement discussions examine the effectiveness of the training provided. (Haley et. 


al., 1995) 
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Raytheon’s software development process valies on pathfinding, which is the 
process of identifying and testing tools, methods, and software components prior to their 
required use on a project in both the development and test environment. Pathfinding is a 
key component of risk management because problems can be identified early, and 
mitigation steps can be taken before there is schedule or cost impact to the project (Haley 
et. al., p. 15). Tools that are new to Raytheon are given extra attention so as to develop 
explicit procedures governing their use and identify peculiar behavior characteristics. 

Raytheon's three-tiered set of documents are consistent with Dalziel and 
Schoonover's (1988) recommendation of clarifying plans as a means of ensuring 
successful change implementation. Their awareness of environmental factors and the 
influence that these factors may have on process improvement activities is in line with 
_ Pasmore guidance on optimizing social and technical systems within the constrairits of © 
the environment. Quantitative measures of process improvements and key process 
reviews maintain compatibility with organizational practices and plans as recommended 
by Dalziel and Schoonover (1988). 

Over the lifetime of the initiative, rework involved in building software bis: - 
undergone a reduction from about 40% of fhe development cost to about 10%. During 
this same period, productivity of the development staff has increased by a factor of 
almost 2.8, and predictability of their development budget and schedule have been 
reduced to the range of +/- 3%. RES’ ability and willingness to analyze the impact of the 
cnitiative in these business-oriented terms has greatly influenced their success in 


maintaining the ongoing sponsorship of senior management. (Haley, et. al., 1995) 
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G. BEST PRACTICES 


Based on a review of related change management theory and software process 


improvement technical reports and papers the following best practices have been derived. 


It seems logical to group the concepts as follows: concepts related to preparing the 


organization for process improvement, concepts related to implementing process 


improvements, and concepts related to sustaining the improvement effort. 


1. Concepts related to preparing the organization for process improvement. 


Detailed assessment of the organization's preparation for change is a logical 
first step in a software process improvement effort. Dalziel and Schoonover's 
(1988) five attributes of organizational readiness provide a framework for 
assessing the challenges of the process improvement effort: 1) history of 
change, 2)clarity of expectations, 3)origin of idea or problem, 4) support of 
top management, and 5) compatibility of organizational goals. 

Scanning the environment is a critical step in understanding the overall 
context of the process improvement initiative. Buy-in of key customers, 
subcontractors, and suppliers is a must for successful change implementation. 
The search conference method detailed in Pasmore's (1988) sociotechnical 
systems perspective is one example of a framework for assessing the 
organization-environment fit. 

Concepts related to implementing process improvements. 

Improvements should be integrated eradually within the context of an overall 
change design. A change design that relates and makes sense of changes 


within the organizational context is a means of combating "change overload". 
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Transition Management Teams, consisting of personnel ceoreceate all levels 
of the organization, provide a feedback mechanism for assessing the "real 
impact" of improvement activities and demonstrate organizational concern for . 
the people involved in the change. They also enhance participation, provide 
linkage across levels and assign responsibility for change management. 

SPI initiatives as projects. Treating SPI initiatives like projects enhances the 
alignment with the organizational goals and increases visibility of the efforts. 
Maximize involvement in the SPI process. By involving as many 
organizational members as possible in process improvement related groups 
(including discussion groups, transition management teams, training groups, 
etc.) we increase communication between change agents, senior managers, 
and practitioners. We also enhance aveanivational awareness and buy-in of ' 


improvement efforts. _ 


3. Concepts related to sustaining the improvement effort. 


Piloting techniques evaluate new technologies and methods prior to their 
implementation organization wide. The concept of an on-going process of 
experimentation to improve the organization is an important eahaiiand for 
establishing a learnin g culture. 


By integrating small process improvements within the project cycle (guiding 


| change by development project experience) we closely link the process 


improvement efforts to the organization's business goals. Additionally we 


"piggy-back" on the existing reward structures and motivations geared toward 
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successful project releases. The process should be reinforced with structured 
project reviews and lessons learned to enhance the next release. 

Training in change concepts. Transforming the organization into a learning | 
organization is a long-term project. It requires a commitment to life long 
learning on the part of the individual and the organization. It is obvious that 
extensive technical training will be required at all levels. However, training in 
concepts of change management and learning organizations is also required to 


institutionalize the movement to change as the norm. 
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il. METHODS 


The research method employed for this thesis is a case study of software process 
improvement efforts at the Financial Systems Activity-Kansas City (FSA-KC). It relies 
on multiple sources of data, including site visits, archival material, personal interviews 
and a survey. This chapter begins with a iescnouon of the research site. Limited 
descriptions of Defense Finance and Accounting Service (DFAS) and DFAS-KC Center 
are provided because activities at FSA-KC are profoundly affected by these 
organizations. The rernainder of the chapter details the data collection procedures cited 
above. | 
A. DESCRIPTION OF RESEARCH SITE 

1. DFAS 

The Defense Finance and Accounting Service (DFAS) was activated on January 
15, 1991, to reduce the cost and improve the overall quality of Department of Defense 
(DoD) financial management through consolidation, standardization and integration of 
finance and accounting procedures, operations, and systems. In December 1992, DFAS 
took over responsibility for all inten and accounting operations, and the associated 332 
installation finance and accounting offices nationwide and began the consolidation 
process. Currently, DFAS (Figure 3.1) consists of a headquarters, five centers, and 17 
operating locations. DFAS is also actively consolidating and standardizing the hundreds 
of finance and accounting systems within DoD. By fiscal 2002, DFAS will reduce the 


number of DoD finance systems from 67 in 1996 to 9. (DFAS, 1997) 
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2. DFAS Kansas City Center 

The successor to the Marine Corps Finance Center, DFAS-Kansas City Center 
(DFAS-KC) continues to coordinate and supervise disbursement of funds in payment for 
all active duty, reserve survivor annuitants, and retired Marines. In addicon DFAS-KC 
provides financing and accounting services to DoD components worldwide. DFAS-KC 
depends on FSA-KC for the development of software to provide these finance and 


Accounting services. The total workforce at DFAS-KC including civilians, service 


members, and Defense Accounting Office employees is close to 1000. (DFAS, 1997) 
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Figure 3.1 Defense Finance and Accounting Service 
(Adapted from DFAS, 1997) 


3. DFAS Realignment 

Four of the six FSAs within DFAS are co-located with the centers that they 
provide software development services for. However, the DFAS command structure (pre- 
April 98) has all FSAs reporting directly to the Director Financial Systems Organization 
(FSO). The FSO provides DFAS-wide infrastructure services including development 


services, operations services, and business management services. Prior to April 1998 


54 





_ there was a "customer-service provider” relationship between the DFAS Centers and the 
FSAs. The Center Director was not responsible for the performance/viability of the FSA. 


See Figure 3.2. 
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Figure 3.2 DFAS Prior to Realignment (Adapted from DFAS, 1998) 





Effective 5 April 1998 control of FSAs was transferred from the FSO to the 
DFAS center directors. The move was initiated to mitigate "customer-service provider" 
problems by placing responsibility for both functions (information systems and financial 
operations) under one manager. Another reason for the realignment is to leverage the 
information systems expertise (which is concentrated in the FSA/FSO structure) across 
the DFAS Center. The FSA Director has become the DFAS Center Director for 


Information and Technology and the FSO has been renamed the Infrastructure Services 
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Organization (see Figure 3.3). DFAS is taking a phased approach to the realignment. The 
new command, control, and staff relationships, and revisions to policy and procedures 
will evolve over the next few months with October 1, 1998 as the target date for 


completion of realignment. 
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Figure 3.3 DFAS After Realignment (Adapted from DFAS, 1998) 





4. FSA-KC 

FSA-KC provides resources and support in performing a variety of software 
development services for the US Marine Corps, and the DFAS Center. FSA’s primary 
software development arenas are service unique pay and personnel systems (for the 
Marine Corps) and “purple systems” or systems that are developed for DoD-wide use. 
(FSA, 1997) The mainstay of the work at FSA involves the scheduled implementation of 


improvements and revisions to software. These “software releases” normally occur in 
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April and October. The organization develops and maintains a growing number of 
automated information systems (AIS). At this time FSA’s major automated cfomaation 
systems are the Marine Corps Total Force System (MCTFS) and DoD's Standard 
Accounting and Budgeting Reporting System (SABRS). They account for 40% and 19%, 
respectively of the organization's software development effort. The current organization 
chart for FSA appears in Figure 3.4. (FSA, 1997) 

FSA-KC is co-located with the director and staff of DFAS-KC Center. However 
prior to April 1998 the relationship between the DFAS-KC ad FSA was a customer- 
service provider iiitle Effective 5 April 1998 control of FSA-KC was transferred 


from the Director FSO to the Director DFAS-KC. 
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Figure 3.4 Financial Systems Agency (Adapted from FSA, 1997) 













5. Fee-for-Service 

FSA is a revolving fund activity. The activity Operates similar to a not-for-profit 
business. Customers are charged a fee sufficient for the activity to recover the costs of 
ernie the service. In this “fee-for-service” caemiags the objective (for FSA) is to 
break even. Unit costs are adjusted annually based on an agreed upon work-load between 


_ the activity and customers. Obviously in this era of shrinking defense budgets, FSA's 
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customers (DoD and USMC) still maintain the traditional view of trying to get the lowest 
price. Therefore lowering unit costs is a high priority at FSA. 

6. Workforce 

FSA has a diverse workforce consisting of civilians, contractors, and military 
personnel. The current workforce at FSA consists of 42% civilian employees, 32% 


contractors, and 26% military personnel. The breakdown of employees is: 


e Civilian: 147 

e Military: 93 

e Contractor: 113 

e Total: 353 ~=(FSA, 19977) 


7. Customers 
The primary customers, of the FSA are the US Marine Corps and DoD. 
Headquarters Marine Corps (HQMC) submits requirements for improved or upgraded 
services via the Manpower Information Systems Support Activity (MISSA), Kansas City, 
which acts as the site representative. DoD initiated modifications and improvements to | 
_ software are routed via the Accounting and Finance Directorates of the DFAS-KC 
- Center. (FSA, 1997) 
8. Organizational Goals: 
The goals of the FSA as stated in the document entitled “FSA Director’s Goals 
and Commitments” are: 
® Provide exemplary customer service. 


e Reduce business costs. 
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e Continue process improvement to reach Level 3 of the CMM for the entire 
FSA. 

e Develop migration strategy to implement the Joint Technical Architecture 

9. History of Software Process Improvement Efforts 

The SPI ‘nfiasnctars for FSA-KC was formed in 1993. The Management | 
Steering Committee and the Software Engineering Process Group (SEPG) were formed 
in July. Membership in the management steering committee includes the director, deputy 
dareetorssenion managers, and the head of the SEPG. The SEPG is an independent team 
with 4 full time members. The FSO, which has a similar SPI infrastructure, eames 
leadership and guidance, isthe in the areas of technical architecture and process 
scapeavenient (FSA, 1998) 

The SPI efforts were initiated in the organization's Marine Corps Total Force 
' System (MCTFS) Division. The initial assessment of the software engineering practices 
was seidliesial on 1 June 1993. The purpose of the assessment was to gain an 
understanding of the division's software engineering practices, identify key areas for 
_ process improvement, and develop an action plan. At that time the division was assessed | 
as CMM Level l. 

A Software Capability Evaluation (SCE) was performed for MCTFS on 
September 16-20, 1996, at which time the division was found to satisfy four (out of 6) of 
the CMM Level 2 key process areas (KPAs). Software Project Tracking and Oversight 
(SPTO) was rated ‘“ unsatisfied. The Software Subcontract Management KPA was not 


evaluated. A second SCE was performed on TFS on January 27- January 30, 1997 to 
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evaluate the SPTO key process area. Having satisfied all requirements for the SPTO key 
process area, MCTFS was assessed at CMM Level 2. 
10. Current SPI Efforts | 
FSA-KC’s software process improvement efforts are ongoing. Currently the 
MCTFS program is assessed at CMM level 2 and the SABRS program is at CMM level 
1. SPI efforts in the SABRS AIS were initiated in 1997 but were quickly overcome by 
events--specifically a backlog of requests for software modification and development in 
the upcoming releases. The FSA-KC organization is in the planning stages of a level 3 
initiative that they expect to complete by October 1999. At that time it is expected 
MCTFS will be at CMM Level 3 and SABRS will be at CMM Level 2. 
B. DATA COLLECTION PROCEDURES 
1. Process Improvement Survey 
A 40-question survey was designed to assess the challenges facing the FSA’s 
process improvement initiative. Survey content was desi ened to capture the following 
key aspects of implementing change as indicated in the literature review. 
e. Level of Buy-in Across the Organization 
e Management Support | 
e Clarity and Consistency of Organizational Goals 
e Alignment with Organizational Goals and Practices 
e Encouragement of Innovation 
The survey was desi ened to be administered to the entire FSA organization during 
the author's second site visit. Demographic data was requested to classify results by 


division, role (senior management, management, and section personnel), software 
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experience, and time employed at the organization. The survey was administered via 
organizational e-mail with the use of a commercial survey software tool. The survey 
software provided a means to collect and Statistically analyze responses. 

Due to a requirement for prior approval by the Union and to other surveys 
scheduled to be administered during the same time frame, the process improvement 
survey was administered a month after the second site visit. The survey was sent out 
er the demographic data. There were 72 responses from a possible 250 civilian and 
military employees for a 29% response rate. Appendix A contains the cotpleie survey. © 

2. Archival Data Collection 

During the site visits volumes of data relating to the FSA organization and its 
software process improvement efforts were obtained. Documentation on general 
. procedures, SPI activities, and Benchmark studies provided valuable data on the 
functioning of this complex organization, the history of its SPI efforts, and its current 
plans for implementation of SPI initiatives. The FSO’s Post Implementation Review of 
Level 2 Implementation (December 1997) provided particularly rich data on employee 
attitudes about process improvement. The survey conse of 16 questions (12 rating 
questions and 4 open-ended questions). The gaseous address issues relative to the 
support provided for level 2 SPI ivies The survey was administered at al] six FSAs, 
and across all levels of the ereanizanons Data from this report are analyzed in the 
chapter that follows. Appendix B contains the spies survey. 

3. Personal Interviews 

Semi-structured interviews were conducted during both visits to FSA-KC. The 


purpose of the interviews was to assess organizational buy-in, senior management 
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support, and general attitudes and perceptions of software process improvement. A total 
of 14 individuals were ciemiewed including: FSA Director, Project Officers for MCTFS 
and SABRS, 3 Division Heads (out of the 7 that were available), 6 Branch Heads (out of | 
13 that were available), and on site customer representatives for the US Marine Corps 
and the DFAS Center. Each interview was conducted individually, lasting approximately 
15 minutes on average, and notes were taken. Interview results were analyzed for 
common themes and unique perspectives. Appendix C lists the interview questions. 

4. Site Visits 

Two visits to FSA-KC were conducted over the course of the thesis research. The 
initial visit took place from 23-27 March 1998 and the follow on visit was from 20-23 
July 1998. The purpose of the initial visit was to become familiar with the organization, 
including its structure, the software process improvement infrastructure, and general 
attitudes/perceptions about software process improvement. During this time the author 
shadowed the SEPG, attending number of meetings on the status of SPI efforts within 
FSA, and also observing the SEPG’s brief on software process improvement to the new 
DFAS Center Director. Additionally, the author reviewed documents and conducted 
interviews. | 

On the subsequent visit, the goal was to focus more specifically on gathering data 
related to the research question: What are the challenges to the software process 
improvement efforts at FSA? Again, the author spent a good portion of the week 
shadowing the SEPG, reviewing documents, conductin g interviews and asking questions. 


These data, along with the more formal interviews are used in conjunction with survey 
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data from FSA and FSO to identify the perceptions of SPI, its purpose, progress and 


challenges. 
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IV. RESULTS AND ANALYSIS 

A. INTRODUCTION 

In this chapter the results of a process improvement survey, archival material, 
personal interviews, and general impressions gained from site visits are presented and 
analyzed. These data are used to extract themes relevant to challenges of the FSA’s 
software process saceabaililiiania This chapter concludes with a summary of the 
results and their implications. | 
B. PROCESS IMPROVEMENT SURVEY 

A forty-question survey was designed to analyze the dimensions of the challenges 
to the FSA’s process improvement initiative. The survey was administered via 
organizational e-mail with the use of a commercial survey sihtivs tool. There were 72 
responses from a possible 250 civilian and military employees, for a 29% response rate. 
Due to the limited sample size and the lack of demographic information, the reader is 
cautioned that survey results may not be representative of the population. The survey is 
of value to the extent that it may be used with other data gathering techniques in analysis 
of the FSA sigiization Survey questions cover a broad range of organizational aren 


and culture issues. However, the most relevant results are found in the responses to 


questions regarding the following issues: 


e Clarity and consistency of organizational goals. 
e Perceptions on the need for process improvement. 
e Compatibility of process improvement with organizational practices. 


e The extent to which innovation is encouraged. 
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Survey questions and statements addressing the aforementioned issues and 
summaries of responses by percentage are provided in Tables 4.1 through 4.7. The 
complete results are provided in Appendix A. 


Responses to questions are rated on one of two five point scales as shown below. 


Selection Rating Selection 

Very Much 5 Strongly Agree 
Some what 4 Agree 

Very Little 3 Disagree 

Not at all 2 Strongly Disagree 
Don't know |! Don't know 


| A level 5 response (Very much or Strongly Agree) is considered a high rating reflective 
of a more definitive or sositive response to the question or statement. A level 2 response 
(Not at all or Strongly disagree) is considered a low rating indicating a more definitive 
negative response to the question or statement. _ 
1. Clarity and Consistency of Organizational Goals 
The questions regarding organizational goals are designed to assess the clarity and 
consistency of organizational goals throughout the organization. Dalziel and Schoonover 
_ (1988) discuss compatibility of a planned change with organizational goals as a primary 
factor in ee an organization ready for the change. A good starting point is ensuring 
that current goals are understood and shared by all. 

The first series of questions (Table 4.1) assess respondents perceptions on the 
extent to which each factor—customer satisfaction, product quality, controlling cost and 
schedule—is a ee of the organization’s success. The second series of questions 
regarding organizational goals (Table 4.2) ask respondents to indicate the extent to which 


each of the aforementioned factors is a measure of success in their immediate work area. 
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Very Much Some what _—‘Very Little Not at ail Don't know 


To what extent is customer satisfaction used 
as a measure of success at FSA? — 45 37 14 


To what extent is product quality used 
as a measure of success at FSA? 


To what extent is controlling cost used 
as a measure of success at FSA? 


To what extent is schedule used 
as a measure of success at FSA? 





Table 4.1 Ratings of Goal Clarity and Consistency in FSA (%) 

Responses to the first series of questions are almost equally distributed with 
respondents indicating that all of the factors are measures of organizational success. 
Between 45% and 48% of respondents gave level 5 responses for cost and schedule as 
measures of the organizations success. Although there were slightly more level 5 
responses for scheaiile as a measure of serait endl success, none of the factors really 
emerges as the primary measure of success. 

In contrast, responses to the second series of questions regarding measures of 
success within the immediate work area indicate that respondents view product quality 
(69% sce ae much”) as a eraiag measuté of success within their immediate 
work area. Both customer satisfaction and schedule also received level 5 ratings by over 
50% of the respondents. Cost is reported less frequently as a significant measure of 


success in the work area. 
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Very Much Some what _—*Very Little Not at all Don't know 


To what extent is customer satisfaction used 


as a measure of success in your immediate work area? - §9 28 


To what extent is product quality used 
as a measure of success in your immediate work area? 


To what extent is controlling cost used 
as a measure of success in your immediate work area? 





To what extent is schedule used 
aS a measure of success in your immediate work area? 52 28 11 1 


Table 4.2 Ratings of Goal Clarity and Consistency in Work Area (%) 
The results in Table 4.1 and Table 4.2 indicate that respondents are more aware of 
the measures of success in their immediate work area than they are of the measures of 
success for the organization. This is indicated by the higher percentage of “Don’t know” 
responses to the first series of — While. this is an expected outcome, the 
difference between respondents perceptions of the importance of product quality as a 
measure of success within their immediate work area and the importance of product 
quality to the success of the organization is significant. The results suggest that 
employees believe that their efforts at ensuring high product quality may be at odds with 
the organization's other goals. Guanes satisfaction and reducing cost are two of the 
stated goals of the FSA Director. (FSA, 1997) However, these factors don't emerge as the 
primary measures of organizational success in this survey. 

The results in Table 4.1 and 4.2 are indicators of the organization’s goals to the 
extent that perceived measures of success are an enactment of such goals. The 
implications are that the organization’s goals are not sufficiently well understood and 


shared by all. Additionally, senior management at FSA may be bypassing a great 
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opportunity to facilitate adoption of process improvement efforts by not building on the 
employee motivation for product quality. The discontinuity between employee 
aercepnon: of measures of success in their work areas and organizational measures of 
success may suggest that employees do not understand how their efforts contribute to the 
organization's success. For example the lower rating of importance of the measure of 
costs at the unit level may reflect a lack of awareness of the importance of this criteria 
and an unclear understanding of unit level contribution to total costs. Moreover, the 
discontinuity = measures of success in the work area and organizational measures 
of success may signal that employees feel that the “product quality” indicator of success 
that dominates at the work unit level is not valued as significantly at the organizational 
level. 
2. The Need for Process Improvement 

| Achieving a CMM Level 3 rating is a stated goal of the FSA Director. Statements 
regarding FSA’s primary motivation for initiating process improvement are designed to 
assess employee perceptions of the need for process improvement. (FSA, 1997) The 
responses shown in Table 4.3 indicate some confusion regarding FSA’s primary 
motivation for implementing process ciprawerhent efforts. It is surprising that only 30% 
feel strongly (as indicated by a “Very much” (5) response) that product quality is the 
primary motivation for the organization’s process imptovement efforts. Additionally, the 


results suggest that a large percentage of respondents don't know what the organization's 
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primary motivation for initiating SPI efforts is. 


Very Much Some what _—*Very Little Not at all Don't know 


In the future a CMM level 3 certification will be required for DoD 


software developers. 25 31 


The requirement to initiate software process improvement efforts 
came from higher headquarters. 


Process improvement efforts will result in FSA producing 
higher quality products in a more timely fashion. 





Table 4.3 Ratings of the Need for Process Improvement 

Bridges (1988) asserts the importance of a clear purpose for a planned change. 
The purpose should align with internal organizational characteristics. The results here 
seem to reinforce the results of the previous section that suggest the connection between 
product quality and process improvement efforts has not been made clear to employees. 
To the extent that employees view SPI as mane from higher authority without 
acknowledging the potential benefits to organizational performance, there will likely be 
greater resistance to the changes involved in SPI implementation. 

3. Organizational Buy-in and Support 

Support of top management is critical at the initial stages and throughout the 
process of planned change. (Dalziel and Schoonover, 1988) Questions of how much 
organizational buy-in/support are aecisned to assess the buy-in and support at the senior 
management, middle management, and section levels of the organization. Responses to 
the questions have implications for the level of siaadicniad and motivation for process 


improvement at the various levels. 
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Very Much Some what _—s*Verry Little Not at all Don’t know 


In your judgement how much buy-in and support for 
process improvement is there among Senior Management? 35 27 13 23 


In your judgement how much buy-in and support for ° 
process improvement is there among Middle Management? 


In your judgement how much buy-in and support for 
process improvement is there among Section Personnel? 





Table 4.4 Ratings of Organizational Buy-in and Support (%) 

The responses (Table 4.4) are very dispersed for each of these questions. This 
shows low consensus. However, the data do not suggest a great deal of organizational 
buy-in/support at any level. The data indicate that the buy-in/support for process 
improvement decreases as 7 go from senior management to middle management and is 
lowest at the section level. This may indicate that the organization's efforts at achieving 

buy-in are aimed at managers and not practitioners. It may also indicate that section-level 
' practitioners are not sufficiently involved in SPI activities. 
4. Compatibility with Organizational Practices 
A process is not likely to be adopted when it is not clear that the change will 
| enhance current practices. Statements regarding eompaubiliy with organizational 
practices are desioned to determine ieaieae to which process improvement efforts fit 
into existing organizational practices. See Table 4.5 for actual results. 

Responses to the first statement indicate that 51% “don’t know” whether the 
CMM framework for process improvement will address important AIS specific areas of 
software fevciopment The remainder of responses are almost evenly split as to whether 
the CMM framework addresses important AIS specific areas of software development. 


While the majority of respondents do not agree that process improvement efforts 


detract from day to day operations (45%), a significant minority (34%) does agree with 
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the statement. Responses to the third statement, that the organization’s preoccupation 
with process improvement has caused neglect of other issues are fairly evenly split. Most 
respondents (50%) disagree with the fourth statement that day to day operations don’t 
leave time for SPI activities, but a significant percentage (35%) do agree with the 
statement. Finally, the majority of respondents agree (48% agree, and 11% strongly 
agree) that SPI initiatives will make their jobs easier. 

The responses to the first question regarding whether the CMM addresses AIS 
specific areas are significant in light of case study findings on the need to recognize and 
adopt general CMM guidelines to unique product/process or organizational requirements. 
The abundance of “don’t know” responses is another indicator of the need for more 
participation in process improvement efforts. Top down change management will over- 
emphasize the generic rather than the more useful locally Seca process changes. 

The responses to the second, third and fourth statements indicate the challenge 
that faces FSA with respect to balancing the piocessinaptavement initiative with day to 
day operations. The responses demonstrate cause for concern that software process 
improvement initiatives may be in conflict with organizational practices and goals. This 
concern reinforces the question of the eee of SPI goals relative to the existing 


performance goals of the organization. 
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Strongly Strongly 


Agree ' Disagree Don't know 


_ Regarding the process improvement framework, there are 
important AIS specific areas that the CMM does not address. 51 


Process improvement efforts detract form day to day 
operations. 


Our preoccupation with software process improvement has caused 
us to neglect other important issues facing the organization. 


Day to day operations don't leave time for software process 
‘improvement activities. 


Pianned software process improvement initiatives will make my | 
job easier. 





Table 4.5 Ratings of Compatibility with Organizational Practices (%) 

5. Encouragement of Innovation 

Statements regarding encouragement of innovation (Table 4.6) are designed to 
assess the organization’s effectiveness in creating an environment that encourages 
employees to actively participate in process improvement. Bridges (1991) believes 
innovation will happen automatically in the neutral zone if Seaple are provided with 
encouragement and support. The extent to which employees believe that their ideas will 
be acknowledged a acted on and that innovation will be rewarded is a key indicator of 
the climate of the organization as it relates to the enhancement of learning and 


innovation. 


Strongly Strongly 
Agree Disagree Don't know 


Recommendations/ideas for process improvement are 


acknowledged or acted on. . 17 


Successful ideas or recommendations for process 
improvement are rewarded. 





Table 4.6 Ratings of Encouragement of Innovation (%) 
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A majority of those surveyed agree that recommendations for process 
improvement are acknowledged and acted upon. However, there is virtually no strong 
agreement with this statement. In addition, a majority of respondents disagree or are not 
aware that there are any rewards for successful process improvement ideas. 

6. Challenges to Process Improvement 

Six statements are provided to gauge employee perceptions of the challenges to 
FSA’s process improvement efforts. The statements cover a range of typical problems 
encountered in process improvement efforts as indicated in a review of case studies and 
technical reports. See Table 4.7 for results. 

The results indicate that respondents view all of the statements as likely 
challenges to the process improvement efforts. However, the responses indicate that 
employees perceive the following as the most likely challenges to the organization’s 
process improvement efforts, based on the aggregate of the top 2 rating categories: 

e Time and resource constraints. 

e The possibility of being overcome by other priorities. 

e Lack of customer support. 


e Insufficient developer training and education 
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Very Much Some what _—s—~Verry Little Not at all Don't know 
Process improvement goals are not integrated into 
currently established FSA goals. 15 38 


Lack of customer support. 33 33 


Time and resource constraints. 


Problems matching CMM requirements to AIS specific 


activities. 
Insufficient developer training and education. 


Overcome by other priorities. 


Table 4.7 Ratings of Challenges to FSAs SPI Efforts (%) 





These results have implications to employee perceptions of the viability of the 
organization’s SPI program. The above factors may be an indicator that employees 
perceive that process improvement is not a high priority of the iseiainediats or that the 
sieanizaien + leadership will not ensure that the necessary resources and support are 

. Sustained. 
- These responses provide a starting point for senior management in addressing 
7 employee attitudes and concerns about the current process improvement effort. The 
possible perception that process frapreveuicie iS a passin g fad will ake gaining the | 
commitment and energy of employees a challenge. } 
C. ARCHIVAL MATERIAL 

The Financial Systems Organization's Level 2 Post Implementation 
Review provides a wealth of data and perspectives on the challenges of SPI. As part of 
this review, a survey was conducted of all six FSAs to capture the perspectives of 
managers, SEPG members, seseiliidia and others involved. Because all FSAs reported 


directly to the FSO prior to April 1998, the FSO was highly involved in the level 2 efforts — 


75 





of all FSAs. Additionally, some aspects of the FSA's SPI program were mandated by the 
FSO. | 

The review, compiled in December 1997, consists of sixteen questions. Ten of the . 
questions ask respondents to provide a 1-5 rating. The other questions are open-ended 
and comments are recorded. Eight of the questions (6 rating questions and 2 open-ended) 
covering concepts most relevant to this thesis are included in this section. The full text of 
the review is provided in Appendix B. Data from the review are analyzed comparing 
FSA-KC and FSO as a whole. 

Table 4.8 lists six questions concerning support mechanisms for software process 
improvement and the ratings provided by members of FSA-KC and by the members of 
the other FSAs. Results of all FSAs (including FSA-KC) are averaged and listed under 
the heading of FSO. For all questions using the numeric scale a rating of 3.00 represents | 
the middle ground of the response. Any rating below 3.00 indicates a disagreement or 


low appraisal, while any rating above 3.00 represents agreement or a high rating. 
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FSA-KC 
N = 50 


Rate senior management support at your FSA for SPI. 3.63 


Rate the info. related to improvement activities that was made 
available to you. : 3.61 


Rate the training you received on SPI initiatives. 254 
Rate the FSO support activities to SPI initiatives (These activities 
include training, FSO SPI conferences, SMS workshops, 


and process definition). 


Rate the tools provided by FSO for SPI initiatives. 


Rate the local SEPG support you received on SPI initiatives. 





Table 4.8 FSO Level 2 Post Implementation Review 

As shown in Table 4.8, support mechanisms internal to the FSAs, senior 
management support, SPI related information made available and training on SPI 
initiatives, received ratings above the midpoint (3.0). SEPG support for SPI initiatives 
was rated high FSO-wide, with FSA-KC receiving codes higher fia the average rating 
of SEPGs within FSO. In contrast, FSO-level support activities received lower ratings 
| than local support; and tools provided by FSO for SPI initiatives received the worst rating 
of the support mechanisms. | 

The responses to open-ended questions shed some light to the low ratings of FSO | 
support. Some respondents commented that the maj si problem with FSO’s approach to 
employing support tools is that tools are selected based on their success within a 
particular automated information system and are then expected to yield Sule success in 
other qutomated information systems. The following worker comments regarding the use 
of software development tools mandated for use by FSO are illustrative of the problem: 

AISs are expected to adopt to tools developed for AISs in different environments 


(i.e. CMIS). Trying to force round pegs in square holes. Attempting to use LRS for 
metrics in ways it is inadequate for. 
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They all have that awkward, home-grown feel. Probably would have been better 
to find tools that suit the needs rather than enforcing tools to fit needs that were 
never envisioned when they were developed. 


CMIS is old and outdated. Written by individuals in a specific site and everyone is 
expected to use. | 


Comments and responses to questions regarding FSO support mechanisms have 
implications on the responsiveness of externally provided support and on technology 
insertion FSO-wide. The implications are that internal support mechanisms are more 
responsive to process improvement requirements than FSO provided support. 
mechanisms, which are expected to support the needs of all six FSAs. This is particularly 
significant as it applies to software management tools. 

Another problematic aspect of the level 2 effort identified in the survey was the 
impact of process improvement on customer relations. When provided the open-ended 
"statement: 'The most significant negative impact caused by SPI efforts was...’ the 
predominant theme was that process improvement activities add to cost (in time and 
money) and stress customer relations. The following comments are representative of the 
theme. 

Need more eusionier commitment atid support for SPI. The program is too 
costly and important to FSO not to gain strong, active, early commitment 
starting at the top of the DFAS management chain. 


Cost too high, cost to implement and maintain. 


Added paperwork that needs to be charged to a customer who already doesn't 
have enough money to get the job done. 


Customer has claimed slower turnaround for impacts and releases. 


These responses reinforce the data gathered in the FSA-KC process improvement 
survey (section B) that point to competition for resources and customer support as the 


primary challenges to the organization’s process improvement efforts. The implications 
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are that customer support may not be sufficiently addressed by the leadership of FSA. 
The expense of process improvement activities in time, resources, and personnel requires 
not only internal support and buy-in, but also an enlightened, supportive customer. 
D. PERSONAL INTERVIEWS 

1. Management Perspectives 

Semi-structured interviews were conducted during both visits to FSA-KC. Due to 
time constraints and limited personnel availability, only managers (Project Officers, 
Division Heads and Branch Heads) were interviewed. Management perspectives are 
provided in question and answer format in this section. The ee to these questions 
are indicators of iano expectations for process improvement efforts, Director 
support for process improvement, organizational goals, and areas for ne within 
the organization! The questions here represent a subset of the eciee questions. The 
full set of interview questions appear in Appendix C. 

a. Expectations for Process Improvement Efforts 
e What concrete observable outcomes will result from process improvement 
efforts? 
e Do the proposed changes make the employees jobs easier of harder? 
The following statements are representative of the responses to these 

questions. 

More structure will facilitate control, accountability, and efficiency. 

| Better quality. ie Production Incident Reports. 
Reduced duplication of effort. That was one of the benefits from level 2. 


The responses echo those found in the FSA-KC process improvement 
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survey and the Level 2 Post Implementation Review. Managers are receptive to the 
concept of software process improvement and have positive expectations. All 
respondents believe that planned changes will make employees ibe easier. 
b. Director Support 
e What does the Director say about SPI? What does he do? 
Managers cited inconsistent Director support for SPI. The following 
statements are representative of the responses to this question. 
The Director talks SPI, but does not peta adequate support. 


He (the Director) wants to achieve level 3... he's not really willing to make 
sacrifices to get there. His #1 priority is to get the release out on time. 


The Director must be held accountable for SPI if you want wed support from him. 
The responses here are an indicator of competition for resources and 
conflicting priorities. In discussions with cencnenis the perception is that the Director 
does in fact value SPI activities. However, within the sencteaats of limited resources and | 
competing priorities SPI initiatives often take a back seat. 
c. Organizational Goals 
© The most important thing to the leadership of FSA i e 
The question of what is the most important thing to the leadership of FSA, 
is designed to assess management’s perceptions of organizational goals. This is 
significant because if the organization’s SPI efforts are to be adopted and maintain 
- consistent support, they must be compatible with the organization’s goals. 
Decreasing cost and meeting the release schedule are seen as the primary 
goals of FSA. All responses to the question of referenced cost, schedule or both. The 


obsession with reducing cost appears to be due mostly to the fee-for-service concept that 
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the FSA works under. The concept of fee-for-service is a major part of the Department of 
Defense's attack on waste and abuse of government funds, by encouraging DoD activities 
to operate like commercial businesses. It provides visibility of fees charged by service 
providers (unit costs), with the ultimate goal of eliminating any non-value added aspects 
of business. Decreasing unit costs is an obvious priority of any fee-for-service activity. 

d. Areas for Improvement 

e What specific parts of the organization provide the most opportunity for 
increases in efficiency and quality? | 

This question is désigned to elicit responses that address problematic 
aspects or areas for improvement within the organization. Managers see the issues of 
customer-relations and requirements analysis as the areas with the highest potential to 
increase the eiveni anon efficiency and quality. The following statements are 
representative of the responses to this question: 


The customer has set unrealistic requirements and schedule demands... SPI is 
seen as delaying the process. 


They (senior management) want it all--cost, quality, schedule--even in situations 
where it is.obvious that there are tradeoffs to be made. This is driven mainly by 
the customer. | 
This is another reference to the importance of the customer in the 
organization’s plans and practices. As Pasmore (1988) asserts, the external environment 
(represented here by the customer) is the ultimate judge of an organization’s survival. 
The comments are also consistent with the Level 2 Post Implementation Review 


comments discussed previously (section C). This demonstrates the problem of customer 


support for SPI is systemic and not particular to FSA-KC. 
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2. Customer Perspectives 

Two customer representatives for the MCTFS AIS were interviewed to assess the 
level of awareness, buy-in and support for process improvement. They were asked the | 
following questions: 

e Have you been briefed on SPI? 

e What are your initial impressions of SPI at FSA? 

e Will SPI initiatives help you? Are you willing to pay? 

Both interviewees indicated that they were not provided a briefing on SPI. Neither 
could pinpoint how FSA’s level 2 efforts impacted the customer whom they represent. 
One representative indicated that FSAs level 2 efforts were transparent to the Marine 
Corps: The second representative remarked "DFAS headquarters says they have to be at 
. ‘a level’. It would be easier to justify the investment if it lowered costs or increased 
quality or something”. 

E. GENERAL IMPRESSIONS FROM SITE VISITS 

During the initial visit to FSA-KC the author had the opportunity to attend a 
‘number of organizational meetings including: The SPUSEPG Meeting between ends 
Deputy Directors and SEPG; MCTFS Project Officers Meeting; SEPG Morning Meeting; 
and SEPG/FSO Conference Calls. Discussions in these meetings provide perspectives on 
the inner-workings of the FSA iad its SPI otto 

In terms of the organization's readiness for software process improvement efforts, 
iitaties are that at the middle management level and above there is enthusiasm for 
SPI. The indications are that SPI initiatives fit into the current organizational practices. 


Specifically managers seem well aware of the benefits of mature software engineering 
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practices and believe SPI activities will enhance day to day operations. However, 
managers are equally aware of the constraints of SPI implementation--limited resources, 
fixed schedules and lack of customer support. This is consistent with data from 
interviews and archival material. 

A high level of cooperation exists between the SEPG and AIS cae SEPG 
members are able to use technical knowledge, experience, and relationships gained in 
part from previous work within the divisions to generate workable action plans that 
reflect the input of serconniel The SEPG is viewed favorably as internal technical 
consultants on SPI issues. This fact indicates possible opportunities for the organization 
to enhance its process improvement efforts by expanding the role of the SEPG. 

While process improvement efforts are somewhat compatible with organizational 
practices they are separate.and sist from the organization’s day to day goals. Process 
improvement efforts have.a loose connection to the organization’s business goals but they 
are challenged by schedule pressure, other priorities, and competition for resources. 
Support for SPI from the Director ebbs and flows with these factors. Recent delays in 
implementation of pigcess improvement activities in the Standard Accounting and 
Budgeting Reporting System (SABRS), due 6 a backlog of key modifications requested 
by the customer, are an example of how intense schedule pressure att derail SPI efforts. 
While the Director is knowledgeable and enthusiastic’about the potential benefits of SPI, 
he appears to be constrained by environmental pressures which many times conflict with | 
SPI efforts. This is consistent with data obtained from personal interviews and the FSA- 


KC survey. 
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There is a perceived emphasis on “making the grade” in the process improvement 
initiative. This detracts from key objectives sought through process improvement: 
improving product quality, and improving customer satisfaction. The “grade” aspect is 
the explicit or implicit view that the primary goal of SPI is the attainment of a 
certification. This is evident in directive statements like “we will be at level 3 by October 
1999.” This approach is particularly detrimental when the date is selected without an 
appraisal of the social and technical issues that must occur to accomplish the goal. 
(Pasmore, 1988) This “grade” centered approach to process improvement strains the 
alignment of SPI with organizational practices and goals. 

While the reward system at FSA does not explicitly address software process 
improvement, it is flexible enough to be used as a tool to reinforce desired behaviors. 
During and subsequent to MCTFS' level 2 efforts, senior managemient rewarded a 
number of individuals for their process improvement efforts. This provides additional 
confirmation of the survey data reporting some use of rewards. It may also suggest that 
this use of rewards is not sufficiently publicized, thus explaining the low agreement on 
_ the use of rewards reported in the survey (see Table 4.6). 

F. SUMMARY AND IMPLICATIONS 

In Chapter II, best practices were derived based on a review of relevant change 
management theory and software process improvement technical reports and papers. 
These best practices were categorized as concepts related to:.1) preparing the 
organization for process hanna 2) implementing process improvements, and 3) 
sustaining the improvement effort. This section summarizes the results of the data 


analyzed and examines the implications of the results along the.same criteria. 
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1. Preparing the Organization for Process Improvement 

Concepts sitive to preparing the organization for process improvement include a 
clear understanding of current organizational goals, communication of a clear purpose for | 
process improvement, establishing compatibility with organizational practices and goals 
and ensuring top management support. Dalziel and Schoonover (1988) discuss 
compatibility of the planned change with organizational goals as a primary factor in 
getting an organization teady for a planned change. A good starting point is ensuring that 
current goals are understood and shared by all. The goals of the Director FSA as stated in 
the October, 1997 document entitled FSA Director's Goals are: to provide exemplary 
customer service, reduce business costs, continue process improvement to reach level 3 
of the CMM, and develop a migration strategy to implement the Joint Technical 
Architecture. (FSA, 1997) In the document the Director commits to using customer | 
service as the driving force for FSA efforts. — 

Results of the data indicate that cost, schedule, and customer service are all 
viewed as measures of organizational success. It is significant that no single factor really 
emerges as a primary measure of the organization's success. In the software industry 
there are constant tradeoffs etwess customer demands, cost, sheiale and srodiict | 
quality. The perception on the part of managers and practitioners that senior management 
does not recognize that there are tradeoffs to be made can lead to frustration and hostility. 
As one interviewee put it: "They want it all--cost, schedule, quality--even when they 
know it's impossible.” 

It is also significant that product quality is not one of the Director's explicit 


primary goals. Product quality is addressed within the context of the other goals. 
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However, the data in this thesis suggest that employees believe that, while product 
quality is the primary measure of success in their work areas, it is subordinate to cost and 
schedule at FSA. This attitude about product quality is not consistent with the attitude. 
that is required to sustain a process improvement effort. It does not harness the employee 
motivation for producing quality software that the data indicate is present at FSA. 
Additionally the discontinuity between employee perceptions of measures of success in 
their work place and organizational measures of success may suggest that employees do — 
not understand how their efforts contribute to the organization’s success. 

Bridges (1988) asserts the importance of a clear purpose for a shite change. 
The purpose should align with internal organizational characteristics. In Chapter I, 
software process improvement was defined as those practices and procedures aimed at 
facilitating the consistent and predictable production of quality software. However, the 
purpose of FSA's SPI efforts is not as clearly defined. Employees are more apt to believe 
that the purpose of FSA's process improvement efforts are to satisfy a requirement from 
higher headquarters or DoD as opposed to increasing software quality. This dilutes 
employee commitment to the effort for at least two reasons: 1) they don't really know 
-how their efforts contribute to the ipanieaionts sia improvement efforts ad Z) 
some may perceive that the process improvement effort is of limited duration, and they 
can endure until it passes. To the extent that employees view SPI as mandated from 
higher authority without acknowledging the potential benefits to the organization’s 
performance, there will likely be erates resistance to changes involved in SPI 


implementation. 
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Dalziel and Schoonover (1988) believe that every leader who initiates a policy, 
program, or practice should assess the compatibility of the initiative with the current 
organizational practices. The mismatch between a change initiative and an organization's . 
daily practices is a key source of resistance to change. The data in this thesis indicates 
that employees believe process improvement efforts are somewhat compatible with their 
current practices. More report compatibility than not, but a substantial minority report a 
difficult balance between SPI and day to day work or see neglect of important areas as a 
result of SPI. This concern eunorees the question of the importance of SPI goals relative 
to the existing performance goals of the organization. The majority of respondents 
believe SPI initiatives will make their jobs easier. Previous success in implementing level 
2 process improvements in the MCTFS AIS is in large part responsible.for the favorable 
_ perception of the benefits of process improvement. The indication is that resistance to 
process improvement efforts is more a function of SPI program management issues than 
technical issues. 

Support of top management is critical at the initial stages and throughout the 
process of planned change. (Dalziel & Schoonover, 1988) To reinforce the new change, 
senior management must send panastenl eeeanes about the importance and priority of 
the change. (Bridges, 1991) The data indicate that members of FSA are receiving mixed 
messages about the priority of SPL In discussions with respondents, the perception is that 
| the concept of improving software processes is important to the Director. However, 
within the constraints of limited resources, competing priorities, and customer demands 
SPI initiatives are often subordinate. The data support the fact that employees are not sure 


that SPI is a high priority to FSA. It is recognized by most in the organization that the 
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Director's support for SPI is constrained by pressures from the customer and higher 
headquarters. Also the iow level of support and buy-in for process improvement at the 
practitioner level indicates that practitioners are not sufficiently involved in SPI activities. | 

2. Implementing Process Improvements 

Internal and external support mechanisms were examined in this chapter. The 
results indicate that internal support--senior management support, SEPGs, information 
and training--is more responsive to organizational needs than external support 
mechanisms provided by the FSO. The costs of standardization of tools and procedures 
may outweigh the benefits. It is logical that a smaller defined organization (1.e., a single 
FSA as opposed to all FSAs) allows for more focused and responsive process refinements 
than a larger organization. 

The ae of software development tools mandated for use by the FSO was 
particularly problematic and has implications for technology insertion. The insertion of 
technology in the work environment whether it is in the form of process improvements or 
support tools is important enough to. warrant evaluation and review. All of the successful 
case studies reported in Chapter II indicate that piloting of processes and tools is a key 
aspect of their SPI efforts. Pilot studies of specific techniques and ileal aa 

subsequent evaluation of the impact of the techniques and technologies prior to their 
widespread implementation will save time and resources in process changes that are not 
effective. The case studies also reported that tailoring process improvement mechanisms 
to locally unique requirements makes for more appropriate solutions. Tailoring and 
piloting may decrease the cynicism that results when employees perceive that they are 


being forced to use tools that are not responsive to their environment. 
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The data from the FSA-KC survey and the FSO Level 2 Post Implementation 
Review indicate that the SEPG is viewed favorably within FSA-KC. This fact offers 
possibilities for expanding the role of the SEPG to enhance the achievement of process 
improvement goals as well as other organizational goals. The added exposure of the 
SEPG may facilitate spintaniication of process improvement goals and organizational 
buy-in. Especially at the sacle level where the data suggests there is a need for 
more employee involvement and participation. 

| 3; Sustaining the Process Improvement Effort 

Sustaining process improvement efforts is a function of encouraging innovation, 
instilling a long-term focus si process improvement, ae managing the environment. 
An organization’s ability to aaprove is a function of the leadership commitment to create 
aculture that invites innovation and continuous learning. (Quann, 1995) As Bridges | 
' (1991) states, people naturally generate solutions to problems they’ ve been living with. 
What they seldom do without encouragement and support is try their ideas. The data in 
the FSA-KC process improvement survey ‘dicate that most employees believe that their 
ideas and casticnnaialialtaas for probes improvement will be acknowledged and acted 
ore banca the majority doesn’t believe that they will be rewarded for innovative 
ideas. This may be remedied by more explicitly linking rewards to process improvement 
efforts and making the rewards more public. 

Continuous software process improvement is a long-term undertaking. The results 
in this chapter jndicate that FSA does not approach SPI with a long-term focus. An 
approach to process improvement that is centered on reaching a certain level (‘making the 


grade’) within a fixed timeline can be counterproductive for a number of reasons. First, 
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‘making the grade’ is not a goal that aligns with other organizational goals. Consequently, 
it does not create sufficient buy-in for process improvement activities throughout the 
organization. Additionally, trying to force change into a prefixed timeline suggests that — | 
the rate of change is not geared to organizational factors. (Dalziel & Schoonover, 1988) 
As Pasmore (1988) asserts, the external environment is the ultimate judge of an 
organization's survival. An overwhelming theme of the results from this study is that 
managing customer expectations and ensuring customer support are the problematic areas 
of FSA's SPI effort. Specifically, the results indicate that the sicioniee is not educated on 
process improvement and its benefits. Customer demands represent constraints and 
opportunities for the organizations that serve them. Successful organizations do more 
than react to the environment; they take steps to transform the environment itself to make 
it more conducive to their well being. (Pasmore, 1988) FSA's process improvement 
efforts are financed to a large extent by its customers. Therefore it is critical that the 


customer be educated on the benefits of process improvement. 
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Ne CONCLUSIONS AND RECOMMENDATIONS 

The intent of this thesis is to facilitate the software process improvement efforts 
of FSA-KC and to generalize the knowledge developed through this research to other SPI 
projects. In this chapter, the high points from the preceding chapters are first summarized. 

This is followed by discussion of the primary conclusions of this thesis and sets of 
recommendations: 1) for FSA, 2) for other SPI projects and 3) for further research. 
A. SUMMARY | 

Chapter I provides background information on the state of software ee 

today. Specifically, software engineering as a discipline lags behind other engineering 
disciplines in terms of the software industry's ability to produce quality software within 
- cost and schedule constraints. Chapter I also discusses the challenges to implementing 
software process improvement citaties, Lack of compatibility of SPI with 
organizational practices and goals, complexity of SPI concepts, and lack of consistent 
buy-in and support for SPI are among the primary challenges to a SPI effort. 

Chapter II reviews three theoretical perspectives on organizational change. The 
work of Dalziel and Schoonover (1988), Pasmore (1988), and Bridges (1991) provide the 
meade for anaives in this study. Although their approaches differ, there is some 
overlap in their coverage of certain concepts. The overarching themes of the literature 
addressed assessing an organization's readiness for change, change implementation 
Strategies, and sustaining change. Using this framework the software process 
‘auamee efforts of four sieaieagons are reviewed. in many cases the success of the 
SPI efforts within these organizations can be attributed to the practical application of one 


or more of the change concepts reviewed. The change theory and the actual _ 
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implementation strategies of these organizations lead to a set of best practices. These best 
practices are categorized as concepts related to: 

e Preparing the Organization for Process Improvement 

e Implementing Process Improvements 

e Sustaining the Process Improvement Effort 

Strategies that prepare the organization for process improvements include 
examining attributes of organizational readiness (history of change, clarity of 
ceaieieiicads origin of idea or problem, support of top management, and compatibility 
with organizational goals) and scanning the environment to assess the organization’s 
preparation for change and understand the overall context of the process improvement 
effort. | 

Effective implementation of process improvements typically include deliberate 
~ design for planned change, the use of transition management teams, treating SPl asa 
project, and maximizing involvement in the SPI program. These measures are designed to 
facilitate the acceptance of process improvement efforts by communicating a purpose and 
_ closely linking process improvements to organizational practices and goals. Additionally, 
these measures enhance participation at all levels of the organization to increase buy-in 
and support. 

Concepts related to sustaining the process improvement effort include piloting, 
integrating process improvements within the project cycle, ol personnel training in 
concepts of change sinseidaianiail The sol of these ieasiires is to create a learning 


organization. 
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In Chapter III, the research methods are discussed, including.a process 
improvement survey, archival material, personal interviews, and site visits that were used 
to assess the challenges to FSA's process improvement efforts. 

Chapter IV provides results and analysis of the data obtained from the 
aforementioned sources. The results are analyzed and examined within the framework of 
the best practices discussed earlier. The results suggest that senior management at FSA 
could do more to prepare the organization for process improvement efforts. Specifically, 
clarifying current ercenizational goals, communicating a clear purpose for SPI and 
aligning SPI initiatives with organizational practices and goals are areas that need to be 
eel 

The results also indicate that internal support mechanisms—senior management 
support, SEPGs, local information and training—are more responsive to organizational, 
needs than external support provided by the FSO. The FSO’s approach to employing 
technical support tools is particularly problematic. Employees expressed concern that the 
tools mandated by FSO do not address unique characteristics of their particular 
_ automated information systems. 

The results Suggest that there are mechanisms in place to encourage innovation. 
However, they may not well publicized. The results suggest there is a need for eater 
involvement in SPI at the section level. 

Finally, the results suggest that FSA is not approaching SPI with a jensen 
racist The acnshadis on auickly making the grade is counter to the continuous process 
improvement paradigm. Additionally, by not educating and involving the customer in SPI 


activities, FSA is not addressing a significant source of resistance to SPI. 
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A summary of the challenges to the FSA's process improvement initiative as 
presented in Chapter IV follows: 

e Lack of clarity and consistency of goals throughout the organization. 

e Software process improvement activities are not aligned with organizational 

goals. 

e External support mechanisms—specifically FSO provided software 

development tools--do not enhance software process improvement efforts. 

e Lack of consistent senior management support for software process 

improvement. 

® Lack of customer buy-in and support for software process improvement 

initiatives. 
- B. CONCLUSIONS FOR FSA-KC 

Balancing customer demands, competing priorities and limited resources is a part 
of everyday life for the senior management of FSA. However, implementing SPI adds 
resistance to change, aiciMoibaiaaaeenient technology insertion, and other social and 
technical issues to the challenges. Past success in attaining CMM level 2, a qualified and 
‘dinate SEPG, and a workforce that is motivated to produce quality products are assets 
in the FSA’s drive for further process improvement. 

SPI is a long-term undertaking, requiring a large investment in time and 
resources. The organization undoubtedly has other daunting priorities—modifications to 
existing automated information systems, designing new automated information systems, 
Y2K, etc. The leadership of FSA must evaluate their plan for implementing SPI and how 


it fits into existing organizational goals, priorities, opportunities and constraints. They 
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must involve all key stake-holders, especially sosincets in the planning. By providing a 
well-communicated vision and a consistent commitment to continuous process 
improvement over the long-term they can better focus the efforts of the employees. 
Tom DeMarco (noted author and consultant on managing software projects) says 
competent software engineers inherently want to produce quality software, it is 
management’s job to remove the spstacies 
C. RECOMMENDATIONS 

In this wees recommendations are organized to address three general areas: 1) 
FSA-KC, 2) other SPI projects and 3) further research. Each is addressed in turn. 

1. Recommendations for FSA-KC 

The following best practices are recommended as solutions to the challenges 
‘facing FSA-KC. | 

Explicit Design for Planned Change 

e Transition Management Teams 

e Piloting 

e . Integration of process improvement activities into the project cycle 

e Scanning the environment 
These practices are discussed below. 

a. Change Design 
The nature of the Defense Finance and Accounting Service organization is 

to ‘iii and to do so frequently. Its mission is to consolidate, standardize, and integrate 
finance and accounting procedures, operations, and systems. The mission guarantees that 


at all levels of the organization there will be changes in jobs, procedures, and systems. 
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There will be realignments and reorganizations. Under these circumstances there are two 
factors which are critical to sustain the momentum of the organization during times of 
change. The first is a solid foundation in exactly what the goals of the organization are. 
Clear organizational goals provide some measure or normalcy in times of chaotic change. 
A firm commitment to customer service and product quality 1s likely to survive any 
changes the organization undertakes. 

The second factor is an explicit design for planned change. It is of critical 
importance that those at the apes levels in the organization be conscious of the 
disruptive nature of change, of the transitions necessary to successfully change, and of 
the value of information during times of change. An overall change design that 
recognizes and addresses the dynamics of change, and make senses of the changes within 
the context of well-established organizational goals can be of value to FSA. 

The change design should include specific plans for communication, work 
design, piloting efforts, adjusting reward systems, providing appropriate training, and 
measuring performance. For example, to enhance communication the design could 
. include mechanisms for pisvidin clarity and visibility to organizational goals. Possible 
vehicles for pronionie organizational goals are vision statements, newsletters, bulletin 
boards, and direct correspondence from leaders of the organization. The plan for work 
design could include assigning roles and responsibility for certain aspects of the SPI 
effort. While these items are common in most organizations, they may not be used to 
ii maximum potential. The use of multiple mechanisms with a consistent theme 


increases the chances that the theme will be communicated throughout the organization. 
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Emerging changes in the organization's functions, procedures, and operations 
should be related to established organizational goals. The leadership of the organization 
should group related changes where possible, and eliminate or minimize the impact of 
unrelated changes. When there are simultaneously occurring change initiatives the 
leadership of the organization should address any apparent conflict between the change 
initiatives and between the changes and the organization's goals. This will mitigate the 
confusion on the part of the employees of balancing the new initiative with existing 
practices and goals. 

b. Transition Management Teams 

AS detailed in chapter II, transition management teams (TMTs), represent 
one strategy used effectively in best practices as part of a planned change design. TMT’s 
consisting of personnel representing all levels of the organization, provide a feedback 
mechanism for assessing the "real impact" of improvement activities, demonstrate 
organizational concern for the people, provide linkage across organizational levels, and 
assign responsibility for change implementation. 

_ FSA should leverage the trust and confidence that the organization has for 
the SEPG to facilitate the TMT function. Data jar@hantet IV indicate possible 
deficiencies in employee involvement and participation at the practitioner level. Because 
the SEPG has direct and frequent communications with the Director they are ideal 
candidates to be the focal point of the transition management function. SEPG members 
could lead TMTs in various parts of the season to facilitate participation, the flow 


of information, and feedback. 
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c. Piloting 

Piloting techniques are used to evaluate new technologies and methods in 
test situations or small segments of the organization prior to their implementation 
organization wide. Piloting techniques can help to mitigate the problem of generic 
technical support tools that do not enhance local SPI efforts, for example. The procedure 
involves pilot studies of specific techniques and technologies on a project, analysis of 
results, and evaluation of plies the techniques or technologies are appropriate for use 
organization wide. It is important to distinguish the organization in this aaa the FSA 
and not the FSO. With the increasing number of automated information systems within 
each FSA it is very challenging to design software tools that will be.appropriate for all six 
FSAs. It is perhaps more feasible for each FSA to design its own software tools based on 
the unique characteristics of its automated information ‘systems and local best practices. 
This approach makes the goals and characteristics of the local organization the driving 
element of change (McGarry et. al, 1994) and will be better received by organizational 
members. 

d. Integration of SPI Activities into the Project Cycle 

By integrating small process improvement activities within the project 
cycle, FSA can more closely link the process improvement efforts to the organization's 
business goals. This process can be reinforced with project-based metrics of performance 
and structured reviews of process improvement activities. Integrated into a project, the 
status of a process improvement activity can be evaluated a often as the project. 
Additionally, we "piggy-back" on the existing reward structures and motivations geared 


toward successful project releases. The primary benefit of this approach is that it keeps 
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software process improvement visible. It keeps senior management involved in the 
process improvement seas This approach establishes an atmosphere of continuous 
process improvement as opposed to bursts of SPI activity followed by lulls which send 
conflicting message about the importance of process improvement. 

e. Scanning the Environment 

Customer ignorance of process improvement activities has the most 
potential for derailing FSA's SPI efforts. Pasmore (1988) states that responses to 
environmental demands can take one of two forms. Reacting to the demands as they are 
presented or transforming the environment so as to eliminate or alter the demands. 
Currently FSA is taking the former approach. The customer who is not knowledgeable of 
the FSA’s SPI program can and has made decisions that impact the organization's SPI 
efforts. The postponement of SPI activities within the SABRS. automated information 
system is an example, The problem is, it is would be of most benefit to the organization 
to take the latter approach. The customer provides the main source of financing for SPI. 
Therefore, it is logical to educate the customer on SPL. | 

The search conference and open systems planning methods detailed in Chapter I 
provide techniques for understanding and assessing the demands and opportunities re 
the environment presents to a particular change initiative. The value of these techniques 
is that it forces the organization to address customer concerns and priorities as they relate 
to the planned initiative. 
It seems that at a el the organization should embark on a campaign to 


educate the customer representatives on the value of software process improvement. A 
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more ambitious goal is the education of the real customers—the United States Marine 
Corps and the Department of Defense—on software process improvement. 
The table below summarizes the challenges and recommended solutions for the, 


FSAs software process improvement initiatives. 


CHALLENGES RECOMMENDATIONS 


*Clarity of *Change Design 
eorganizational goals. 


Alignment of SPI with— eTransition Management Teams 
*organizatonal goals. 


‘Technical support does ¢Piloting and Pathfinding 
enot enhance SPl.efforts. 


‘Inconsistent senior eIntegration of process improvement 
management support. activities into the project cycle. 


. ‘Lack of customer buy-in 
. . s * 
eand support. 4 eScanning the environment. 





Table 5.1 FSA challenges and recommendations 
2. Recommendations for Other SPI Projects 
It is likely that a well-defined process will help most software development 
7 projects to ssinctiaih By keying on the unique software characteristics of the local 
organization process improvement efforts will more likely lead to product improvements. 
Therefore it seems that the key challenge in implementing software process 
improvements is tailoring the effort to the needs of the organization. 

Software process improvement is a process of organizational change. The CMM 
does not address issues of a ge management. Leaders seeking to implement SPI should 


study change theory because ultimately the job of improving processes comes down to 
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changing the ways that people do things. The change leader must understand and 
manipulate the human and organizational aspects of Shatiae: 

3. Recommendations for Further Study — 

As noted in Chapter IV, the process improvement survey was administered 
without the questions regarding demographics. It was the intent of the author to request 
demographic data that indicates organizational level (Senior Management, Management, 
or Section Personnel), and the division that the respondent works in. Administering the 
survey with the benefit of demographic data will allow for many interesting comparisons. | 
For example the results could be analyzed for trends and gaps across organizational 
levels and across the divine within FSA. Additionally the survey could be 


administered at different FSAs to assess the issue on a broader scale. 
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APPENDIX A. PROCESS IMPROVEMENT SURVEY 


21 July 98 


DFAS-KC 
MEMORANDUM FOR DISTRIBUTION 
SUBJECT: Software Process Improvement Survey 


Attached is a Software Process Improvement Survey that each FSA employee is requested to complete. 
This survey contains questions about the software process improvement initiative 
currently taking place within the organization. The questions ask about various aspects of 
the organization and the process improvement effort. 


Survey results will be analyzed by Captain Wendell Bazemore; a student of the Naval 
Postgraduate School, for the purpose of facilitating the on-going process improvement 
efforts. 


Survey responses will not be associated with any employee. Limited demographic data 
will be used solely to develop pooled information for the survey categories. 


William G. Head 
Director for Information 
and Technology 


Attachment: 
As stated 
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Demographic Data 


1. Division: 


2. Role: 
[ ] Senior Management (Project Officers, Division Heads) 


[ ] Management (Branch Heads) 
[ } Section Personnel 

3. Software Experience yIs. 

4. Years at IT&D 


5. [] Civilian [ JMilitary 


Not at all Very Some Very Don’t 


Little What Much Know 
1.To what extent is customer satisfaction used as a 
measure of success at FSA? 
Customer Satisfaction [] 1] [] (] [] 
Quality | [] [] C] C] (] 
Cost * [] C] (] C] [] 
Schedule ] {] [] [] [] 
2. To what extent is each of the following used as a measure 
of success in your immediate work area? 
Customer Satisfaction [] f] [] [] [J 
Quality 1 of 8 OO 0 
Cost : fl oO fF Of 0 
Schedule 1) tf) ff Of 0 
Strongly Disagree Agree Strongly Don't 
Disagree Agree know 


3. Restructuring initiatives within DFAS have had a 
positive affect day to day operations at FSA 


0 ff Oo 0 


4. Restructuring initiatives within DFAS have had a positive 
affect on Software Process Improvement efforts at FSA. (] [] [] [] [] 


5. Give your best estimate of 
the organizations 
current software process 
maturity level 
Level 1.( Initial) Few process are defined. Success depends on 


individual effort. ; 
_ Level 2 (Repeatable) Project management processes are established. 
Realistic project commitments are based on previous projects. 
Level 3 (Defined) All projects make use of an organized, documented. 
and standardized set of activities that are institutionalized_ 
throughout the organization. 
Level 4 (Managed) Detailed metrics are collected for both process and 
quality, and used to quantitatively manage software processes. 
Level 5 (Optimized) Continuous process improvement is achieved 
through metrics and feedback. New ideas and technologies are 
‘constantly tested. 


None Little Some Very Don't 
Much Know 


6. In your judgement, how much buy-in and support 
for process improvement is there among: 


Senior Management [] [] {] [] [] 
Middle Management . [] [] [] [] [] 
Section Personne! SY [] f] [] [] 


Strongly Disagree Agree Strongly Don't 
Disagree Agree know 


7. Recommendations/ideas for process improvement 
are acknowledged or acted on. 
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8. Successful ideas or recommendations 
for process improvements are rewarded. 


9. To what extent are process improvement goals 
clearly stated and well understood. 


10. Regarding improving software development 


processes, 


we know what needs to be improved but we need 
more guidance about how to improve it. 


11. About how much of your work-related time did you spend 
over the past year on process improvement related activities? 


12. Regarding the process improvement 


framework, 


there are important AIS specific areas that the CMM 


does not address. 


13.Process improvement efforts detract from day to day 
operations. 


14. Our preoccupation with software process improvement has 


caused us to neglect other important issues facing the 
organization. 


15. Day to day operations don't leave time for 
software process improvement activities. 


16. Planned software process improvement initiatives will 
make my job easier. 


Questions 17-19 relate to coordination and interaction 
between functional areas. 


17. People identify more with t 


functions 


than the fina! product. 


heir specific tasks and 


18. Everyone knows how their work will affect the work of the 
next person or the quality of the final product. 


19. Boundaries between sections or departments interfere 
with solving joint problems. 
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20. To what extent do the following statements 
accurately 
reflect the primary reason for the software process 


improvement effort at FSA: 


In the future a CMM level 3 certification will be 
required for DoD software developers. 


The requirement to initiate a software process improvement 
came from higher headquarters. 


Process improvement efforts will result in FSA producing 
higher quality products in a more timely fashion. 


21. To what extent is each of the following likely to cause delay — 


or failure of the current software process improvement 
effort at FSA? 


Process improvement goals are not integrated 
— into currently established FSA goals. 


Lack of customer support. 


Time and resource constraints. 
Problems matching CMM requirements to 
AIS specific activities. 


Insufficient developer training and expertise. 
Overcome by other priorities. 


. 22. Which statement best describes your attitude 
towards change? 
"I'm not sure how to change; besides things are going 
pretty well -- maybe we don’t need to change." 


“Everybody gets frustrated by change; but you just 
have to ignore the frustrations and push ahead 
or you wont get anything done.” 


"You need to change; you have to change; in fact, 
I'd get bored if we weren't trying out new things.” 


"We change so much around here that we never 
really master things." 


23. Grade the organization (A, B, C, or D) in the following areas: 
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Customer Satisfaction 
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APPENDIX B. FSO SPI Program 
Post Level 2 Implementation Review — 


December 1997 
Respondents: FSACL FSACO FSADE FSAIN FSAKC FSAPE 
Management: 2 3 1 1 12 7 4 
SEPG Members: 4 2 6 2 4 4 
AIS Practitioners: 9 7 48 30 28 8 
Support Division: 2 2 6 9 7 3 
Other: 13 4 0 
Total: 17 14 71 66 50 19 


The Questions: 


1. Please rate the senior management support at the FSO for SPI (FSO senior 
management includes FSO Director, Deputy Director, FSO-HQ) (average score on a scale 
of 1 to 5): 


SEPG AIS Support 
FSA/DSE Total Managers Members Practitioners Division Other 
FSACL 3.07 3.67 2.75 3.40 3.00 
| OL 1 DK/NA 
FSACO ° 3.75 | 3.33 4. 3.85 . 
2 DK/NA - 
FSADE/ 2.96 2.18 2.33 3.32 2.67 
DSE-MP 5 DK/NA ) 
FSAIN 3.15 3.7 4 3.24 © 2.44 2.9 
2 DK/NA 9 DK/NA 7 3 DK/NA 
FSAKC 3.63 4.00 3.5 3.21 3.33 2.67 
1 DK/NA 4 DK/NA 1 DK/NA 1 DK/NA 
FSAPE 3.53 3.0 4.5 3.5 —3.0 
2 DK/NA 
Comments: 


e Senior management is providing impetus and recognition for SPI. 


e Although the rating is desired, CMM principles are not supported in decisions made. 

e The SP! Program could have been quicker and smoother if the customer had been brought 
| “on board” sooner. The directors and AIS managers would have had an easier job working 
with their AlS Program Manager counterparts if support and direction from DFAS-HQ were 
more visible earlier in the program. The lesson here may be that even though we were 
changing FSO processes and procedures, they have an impact on customer processes and 


perceptions of our support. If they are opposed to ou 


difficult. 


\ 


r direction, they can make our progress 


e Asan organization, many respondents believed we were attempting to implement too many 
“programs” simultaneously (i.e., SP!, CMIS, metrics, LRS, FPA, etc.). These programs were 
competing for the same resources’ time - along with developing and implementing software 
deliverables. Many people had “change overload.” This may mean that we must be aware of 
the number and timing of programs/initiatives we are implementing; we have limited 
resources that must also product customer deliverables. 

e As anorganization, many respondents believed we stumbled along too much in the 
beginning of the program and changed directions too many times (SMS versus CMM focus, 








implementing SMS task level versus procedure level, etc.). Maybe this is where piloting 
initiatives would benefit us. 

Could have provided better control and oversight, funds, and training. A detailed plan would 
have been nice. The “just do it” attitude hasn't gotten us very far in the time the program has 
been in progress. 

| believe that they support the SPI! project primarily because it is initiated from the FSO HQ 
level. However, being so far removed from the actual day-to-day work, I’m not sure they are 
aware of the impacts of the SP! efforts. 

They do not seem responsible for the amount of time spent doing all the administrative tasks. 
The actual work takes second place to all the reporting and auditing. 

A great deal of effort was put into the initiative, unfortunately it was not well planned out in the 
very beginning. A heavy price was paid in later efforts due to this. Also, customer and FSA 
personnel buy-in to the basic principles of SP] was never achieved. 

Comments from FSAs are gnored by FSO. Experience of FSA SEPG members is not 
utilized by the FSO. 

From my perspective, it seemed to be adequate although | was not actively involved with 
FSO Senior Management. 

FSO Senior Management support: | believe that the FSO staff have ssackienti: 
demonstrated a real commitment towards achieving SPI objectives. Mr. Burke’s visit was an 
indication as to the significance in which the FSO views SPI activities. 

| don't know who FSO managers are or what they do. 

Mr. Burke has always presented a reasonable presence on SPI matters. Others in the FSO 
hierarchy became too focused on achieving the grade rather than building the necessary 
framework for continued improvement. 

The executors of SPI at the FSO HQ level approach SPI as a top-driven program to be 
imposed rather than harnessing and focusing the grass. roots desires and ideas for 


improvement. 


I’ve seen too many examples of FSO’s SPI office abandoning processes, like CCBs for the 
SMS or CMIS, because of time pressures. 

| haven’t seen much direct support. 

Often, FSO HQ does not provide feedback to comments or concerns from the FSA. 

Lack of commitment based on resources and funding. 

| haven't had direct contact, but support appears to be very good. 

No complaints here. We knew the SPI program was backed up fully, all the way to the top. 
SCEs would be more believable if done by external teams. 

Policy statements, SMS support, workshops were beneficial. 


At an AIS level, FSO support is not evident. 


At the AIS level, support of the FSO senior management is not clearly evident. 
Why not bring in advisors experienced in SPI who can work with us regularly. 
Barely visible in the background. 

Not aware of FSO involvement. 

We see no evidence of their involvement. 

| really do not know how the powers work to rate. 

1 don’t see any support from FSO. 

Inconsistent, overpowering, and burdensome at times, then absent and lacking. 
Support by FSO has not been visible to me other tan a memo stating their support. 
Time was allotted and classes were held with good presentations. 

Do not know what they have done. 

Not being privy to the thoughts of FSO senior management, | cannot answer this question. 
Have not been involved yet in any training. 

Unknown. 

| don’t know who these people are. 

| had no contact at this level. 
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e Not sure if senior management actually understands what the full ramifications (such as the 
amount of documentation) actually are. 

e Initial support appeared to be marginal - it improved once realized that success depended on 
team effort. 


Their views/support do not reach my level. 


e Changes in direction - no clear vision - training and funding haphazard. 

e FSO senior management is viewed as being a proponent of SPI and understanding the need 
for FSO activities and applications to implement a SPI program. 

e SPI has proceeded with a “Report Card” mentality. Little commitment to REAL process 
improvement beyond a few assessments. 

e The Program its supported from the top down. 

e FSO senior management provided continuous vocal and administrative support to the SP] 
effort. 

e FSO management has spearheaded the SPI program from the beginning, providing initial 

‘ funding for improvement initiatives/training. 

e Not enough funding/people to correctly implement at FSA level. 


2. Please rate the senior management support at your FSA for SPI (Site senior manager 
includes the Director and Deputy Director) (average score on a scale of 1 to 5): 


SEPG AlS Support 
FSA/DSE Total Managers Members Practitioners Division Other 
FSACL 3.69 4. 3.25 3.8 3.75 - 
FSACO 3.86 3.67 4 3.86 4 = 
FSADE/ 3.58 4.0. 2.83 . 3.57 3.67 
DSE-MP 1 DK/NA 

FSAIN 3.35 3.58 4 ~—«3.52 | 2.67 3.16 

5 DK/NA 1 DK/NA 
FSAKC 3.63 3.86 4.00 3.73 3.29 2.67 

6 DK/NA 1 DK/NA 
FSAPE 4.06 4.25 3.75 4.17 4.00 
Comments: 


e Program is supported by local management 

e Need more active sponsorship earlier and more frequently by the Division Chiefs and 
Directors (i.e., regularly scheduled status: updates from all managers involved with SPI, 
requiring action and results, etc. - in addition to the Director's all-hands meetings). The 
lesson here may be that Division Chiefs and Directors also need education on implementing 
change in an organization. | 

e Seems like we are still looking for ways not to do things rather than better ways to do things. 

e Once again, | believe that senior management supports the SPI effort because of where it 
originated. However, at the local level, we could use more support in the area of resources to 
pertorm plus support the procedures and process. 

e Senior management participated in SPI as requested by FSA personnel. 

e FSA management keeps SPI an elevated concern. 

e They need to remain proactive instead of reactive. It would help provide a clearer 
understanding and buy-in for the entire organization and hopefully less resistance. 

e | believe that FSA Senior Management has consistently placed SPI activities at the top of the 
FSACL priority list. The high priority given to SPI objectives helped to communicate 
throughout our organization senior management’s commitment towards achieving Level 2. 

e - Additional support was made available from the systems support division for some tool 
development. 
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e While the FSA management did not fully embrace improvement efforts initially, they 
consistently provided resources and openly supported the program. 
e SPI became much more important when others looked like they would achieve Level 2 first 
and when this became a performance evaluation item. 
Support was consistently high, often at the expense of developmental efforts. 
Consistently try to educate us on what is happening and issues we need to learn about. 
Seems to support this effort positively. | 
SCEs would be more believable if done by external teams. 
Hesitant to pay for effort, and some lack of direction - perhaps including quality in estimates 
would improve this. 
e Too much emphasis on Level 2 and not enough on improving our business. : 
e Seems that FSA management lost site of goal - process improvement. Goal is now receiving 
Level 2 with no concern given to whether any improvement was realized. Mostly lip service. 
Hardly any training - not resourced. 
Seem to want the certification level, but not the quality. 
Seem to support process-driven activities, training, documentation. 
Concern is with reaching Level 2, not improving process. 
The people from division level down are exceptional. The higher FSA levels are sufficient. 
| don't see any support from senior management. 
Supported efforts but occasionally lacked answers. 
Support has not been visible, but they have not been a hindrance either. 
There was sincere desire to achieve Level 2 of CMM. 
Do not know what they have done. 
Not being privy to the thoughts of FSA senior management, | cannot answer this question. 
Have not been involved yet in any training. 
Constant emphasis by sénior management on SPI. - 
Support is there only because they were told it has to be there, not because they believe in 
the process. 
e We were date-driven by director. Not ready for evaluation, but had to have one when he 
said, not when we were ready. 
Bob and Joe have very strong support for SPI, and | feel this has helped our organization. 
Conflicting goals - crisis management - no support of SPI personnel - no awards, rewards, 
acknowledgment - no funding - no training - no people - but it still must be in place (SPI and 
. technical issues). 
e FSAPE senior eceleiee has taken an active role with implementing a successful SPI 
‘program. . 
e Provided resources for SPI. 
Funding has been provided, but the previous Deputy Director srevided minimal support to 
SPI and SEPG activities. 
e Director supports SPI, but could participate more actively. Previous Deputy Director primarily 
supported only in the last year. 
e Acouple of “booster” meetings would have been helpful. At least as much as for CFC! 
e We were dedicated to achieving Level 2. 


3. Please rate the information related to improvement activities that was made available to . 
you: — | 


SEPG AIS Support 
FSA/DSE Total _§ Managers Members Practitioners Division Other 
FSACL 3.56 3.67 3.75 3.40 3.50 - 
FSACO 3.79 4.33 4 3.43 4 - 


FSADE/ 3.08 2.73 2 3.41 2.5 
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DSE-MP | 1 DK/NA 


FSAIN 3.13 3.75 3 3.27 2.44 2.75 

3 DK/NA 1 DK/NA 
FSAKC 3.61 3.86 3.5 | 3.72 3.43 2.67 

3 DK/NA 1 DK/NA 
FSAPE 3.94 3.75 4.0 4.0 ; 4.0 
Comments: 


e Handbook and branch meeting weekly topics were most helpful. 

e The SMS training was of little value and the SMS is not user friendly and is difficult to read. 
The AlSs started achieving “real” progress when they relied more internally to train 
themselves (likely on their documented processes, procedures, and mechanisms). Our 
lesson may be two-fold: 

1) The FSO standard processes might be more readily understood with fewer words - more 
graphics and more templates 
2) Guidelines are needed. However, more enthusiastic patticipatien occurs when people 
have 

more control in their direction. 

e The information was sufficient but what was really necessary was insight to the information 
that was available. 

e Our main information for SP! was the SMS. This document was often difficult to understand 
plus difficult to tailor to our system. In one specific case, the SMS did not address a 
necessary CM CMM component in enough detail to determine what was needed. 

e Information, when provided, was good. However, communication of information was 
sporadic and not very timely. This was a great source of frustration to t hose who tried to 
further the SPI initiative. 

-e The SEPG is extremely well run and. organized. Information is always disseminated to all 
concerned as soon as it is received. 

° Early on we were kept fairly current and then little by little, | began to slow until currently we 
receive no information at all. 

e The CMM, SMS manuals and the vast array of training materials helped to establish a sound 
foundation for developing and revising our software development processes. 

e FSO HQ was inconsistent in providing information. While it improved over time, many 
important issues are still not communicated to ALL affected individuals. 

e | feel well informed in all areas of improvement activities. SMD and particularly Pam Cromley 
did an excellent job of getting the word out and keeping everyone informed. 

e Information was good but sometimes came to us in a round about way. of 

e SMS too big, bulky and dulled down way too far into the software development procedural | 
routine. When emphasis shifted to following the CMM, much more relevant and user friendly. 

e SPI documents made available, plus sharing of information within FSA. 

e The SMS is alll am aware of. 

e Not sure what information was made available. If not sure about that, it means that 

communication may be breaking down. , 

e SMS was supposed to get us to Level 2 - It did not. 

e Seem to be behind the curve, trying to catch up. Forcing, hurrying the process will not 

improve it. 

e’ Haven't seen any material of this nature. 

e Can't get the materials. 

e tis overwhelming. Umpteen versions of multiple documents. The tail wags the dog. | 

cannot keep track. 

e Co-workers seem to be saddled with responsibility of supporting/establishing SPI efforts with 

little or no support from senior management. 

e Generally tempered with pragmatic solutions. 

e Certainly was plentiful. 
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Too wordy. Would be more helpful to have visual materials to relate information. 
It was presented in a way that we do our work. 

Each of us given our own copy of guide to follow. 

Have not been involved yet in any training. 

Unknown. 

Enough information has been given to have a basic understanding, but not enough to do the » 


work correctly. 


SE] has it down to a science. 
New information hard to get - need news flashes, bulletin board, or a formal distribution 


system starting with FSO. 


The SEI information proved quite helpful. 

FSO provided excellent information through UPDATE, SPICON and Email. 
Interaction with other FSAs and the FSO has been invaluable to the SEPG. 

Normal workload doesn’t allow us to keep up with this information on a regular basis. 
Meetings were held and reading material made available. 

Report was very good. 


. The most significant improvement caused by SPI efforts was: 
e estimating process 
e use of repeatable processes for SCM and project management 
e documented, accessible, well-understood processes and procedures (improvements 
in approach center more on the CMM versus SMS) 
reviews helped identify questions and issues earlier 
tracking has improved 
better support for firm fixed prices and schedules 
better control | 
the set up of the CM’- with so many pieces and parts involved in any given release, it 
is really nice to have a central area of control 
Better tracking and oversight 
metrics, better collection of data for oversight and reporting 
institutionalizing the practice of having programmers document the location of their 
changes plus putting those changes on a release PDS. If any programmer is not 
available on implementation day, any other programmer can locate the program 
changes plus implement them 
e the stabilizing and reliability obtained in the customer testing region. We now are 
aware of all changes being operated on, all changes being tested as a package. 
Also, ‘baselining of requirements and software was beneficial. 
enhancement of project management activities. 
system releases and bounding of requirements. 
getting a large number of employees to at least learn about SPI and getting them to 
accept and work in a release mode. 
¢ management has an effective way to show all work effort related to a particular 
release. 
e asense of organization about what is in a release and when it will be implemented. 
This begins with early planning and builds from there. 
e better.control with configuration management. Clearer picture of the status of 
projects with PM tracking and oversight process 
e | think it organized procedures and provided a clear and concise way of doing 
business along with the establishment of data archive and feedback thus providing 
continued support for future projects 
e project planning activities are better defined and communicated: 
e software development activities have become more uniform 
e software requirements 
e system components (Cls) are more accurately defined and documented 
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e the project release cycle allows more time for requirements, programming 
and testing to all do a better job 
e implementation activities have been improved and better documented 
® communication has been improved through the FSA as well as with our 
customers 
It has enhanced our coverage for implementing changes when people are out by 
creating a more formal procedure, although we were addressing that as a risk before. 
documentation of processes and centralizing of paperwork 
A unification of effort within the targeted system. SPI helped break down some 
boundaries while addressing a common issue. The same type of coming together 
has begun at the FSO level. 
Institution of standard, formal estimating methods, and a more formal process for 
negotiating, scheduling, and tracking system releases. 
Bringing structure into a chaotic environment. Documenting and producing 
processes was greatly needed. | 
Attaining Level 2. | 
Single point of entry. 
In the requirement management area. 
PANAPT 
Getting people to think outside “the box” of their world and focus their collective 
energies in a united front to increase the quality of software developed and 
maintained for their customers. Discussion and action to improve processes. 
Passing information to practitioners needed to achieve Level 2. 
More control over projects, errors caught earlier in the process. 
Processes in place that force us to pay more attention to the products earlier | in the 


life cycle. 


A heightened awareness of fas flow of work and more accurate measuring of SCRs. 
Repeatability! Manageability! Consistency! | 

Openness to process and improvement and therefore change - improved 
communication resulted in quality products. 


Developing standard procedures for AIS 


Communication and coordination 

Establishment of a goal/direction to work toward - through group efforts, definition of 
how we would reach goals 

Communication outside division boundaries has increased 

Documented procedures 

Getting CMM process underway with several AlSs reaching the Level 2 plateau 
People organized in their jobs - they are more accountable - SPI requires everything 
to be documented and followed 

Awareness, assessment, training 

Don’t know 

The knowledge of the SEI’s CMM process improvement 

Getting the requirements from the customer in writing 

Systematic development/maintenance efforts - repetitive 

Repeatable processes used by everyone 

Reviews seem to catch errors earlier 

We will meet Level 2 requirements 

Getting clearer requirements for change requests - improving understanding between 
user and support groups 

Documenting many of the procedures already in place - providing more uniform 
adherence 

Standardized and repetitive processes have increased productivity - better 
understanding among branches, functionals, and technicals - improved 
communication between FSA centers. 
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Producing a better product the first time 

Crucial procedures needed to make changes are thoroughly defined/documented - 
work more orderly and repeatable 

Quality 

More knowledge about what affects a project’s progress - increased communications 
among technical, environmental, management, etc., members 

Improved our awareness that most of what we do is already Level 2 and repeatable - 
enhanced quality 

More documentation and tracking of software changes 

Documentation on the project 

Standardization 

Gets entire team involved in development efforts 

Proposed/actual implementation of SMS and development of CMM to bring every AIS 
up to Level 2 and beyond 

Documenting on going towards processes and procedures 

Awareness of need to document processes | 

A focus on documenting processes and then improving them 

Have not been involved with any SPI efforts to date 

Don’t know 

Better quality 

Better impact analysis 

Good planning efforts which includes start to end management - good training plan - 
good communication 

Process definition 

Formal reviews 

Managernent of requirements and project management improved drastically 

On schedule - on budget 

Strengthen the team effort and document our procedures 

Reduction in errors/problems in the finished Prune 

Testing procedures 

Communications 

Requirements and design are more completely documented 

Getting business practices documented - established standards for all software 
engineers to follow and a guide for reference 

Worked as a team and all are doing the same way now 

Proactive instead of “fireman” efforts for a highly active production system 
Documentation of existing processes and insurance everyone understands and uses 
the processes 

The micromanagement of processes has caused potential problems to arise and be 
fixed before they become a major issue 

Uniformity in process 

Documentation of processes and projects - we can teach others more easily, move 
personnel, and review processes for problems or improvements 

Business processes which had been used for some time were documented. 
Employees were made aware of current approaches for evaluating system 
development projects. 

Discipline in developing software, scheduling, sizing. 

More working-leve! people were made aware of need for process improvement. 

An awareness of process improvement and | truly believe a desire by practitioners to 
improve. 

Two major AlSs achieved Level! 2 on first evaluation. 

More process awareness, more team orientation, better communication. 

The roles and responsibilities of other SEPG members are known and understood. 
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Development of training plans. 

Documentation of the DCPS process. 

More individual involvement in the total process. 

The current release of the software is a much better and siganel product than would 
have been possible without SPI. 

Opened communication between different levels. 

Consistent methods for handling both routine and emergency SCRs. 

Better Quality Assurance and development of a software process handbook for easy 
access to materials/procedures used throughout the project. 

People are more process oriented. 

Better communications on the project, better understanding of goals and objectives 
and better quality product. 

Regimented software engineering practices iiplemanted: a standard, organized, 
measured method of work. | 


5. The most significant negative impact caused by SPI efforts was: 


increased release time and ability to make changes quickly 

emphasis placed on process at the expense of “nuts and bolts” of 

producing software 

too much paperwork 

need more customer commitment and support of SPI. The program is too costly and 
important to FSO not to gain strong, active, early commitment starting at the top of 
the DFAS management chain. 

The various support tools were not very user-friendly (CMIS, SMS Survey Tool, 

LRS). We needed more top-down customer management support of CMIS. . 

Too much additional (and sometimes duplicated) paperwork and meetings (preparing 
for and developing status reports to be reviewed by multiple levels of management - 
from branch chief up to FSO-HQ director). 

Lines of code projections - | really don’t know how our projects were estimated prior 
to SPI, but we appeared to get things done within budget. 

Increase in costs 

The perception that not achieving the next level after the first audit is a failure. A plan 
that did not emphasize success or failure but long-term improvement would have 
avoided a lot of negativity and bad feelings that resulted from the assessment.: 

The overhead. A tremendous amount of overhead has been added to our day-to-day 
business operation. At times the process overshadows plus inhibits the 
implementation of a quality product. 

There is too much administrative overhead, i.e., too many forms to fill out, too many 
checklists, etc. All these aa have not produced a better quality product. 

Staff morale 

Sense of frustration due to jack of buy-in to the basic principles of SPI. This affected 
proponents and detractors of SPI alike. 

Cost too high, cost to implement and maintain 

The amount of time spent on determining how to implement the CMM, even though 
we started with the SMS 

COST, COST, COST! FSO has an unreasonable view on funding for 
implementation, especially for small systems. 

There is a huge “overhead” and much of this has minimal pay-back. There are many 
things we are doing just because they are necessary to be Level 2. 

Increase in cost for SPI efforts 

The general feeling is of an overwhelming additional workload from the required 
record-keeping and tracking of projects. Many feel it is unproductive and taking away 
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time that should actually be spent programming and is in fact jeopardizing not only 

the project, but the entire organization. 

The negative side is that software modifications require a greater amount of time to 

complete. 

SPI has added a great deal of overhead with little value added 

Paperwork gets in the way of the actual projects. 

Time-consuming up front. 

Resulted in duplication of effort, entering similar information into several systems. 

A lot of effort was spent formalizing processes without improving them. Addressed 

the “book” solution rather than the needs of the AIS. Changes were imposed rather 

than drawn from the AIS. There’s a lot of resistance at the practitioner level in those 

areas where they do additional work without apparent benefit. 

With the pressure on to make SPI-related deadlines, competition for resources within 

the FSA has been bitter. While | have heard some very positive comments from 

some developers, many still see SPI as a drain on their dwindling time. 

| did not detect any negative impact caused by SPI efforts. Every effort made toward 

improvement is a positive effort. 

That it was implemented nearly concurrently with fee-for-service and was used as a 

“standardization mechanism.” 

Great time involved doing process meetings when a mission is time-critical. It 

appears at times we could be better served just doing our job rather than attending 

meetings. 

Initial resistance, however, it seems most employees can see a benefit now. 

Stress due to unclear requirements. 

e Manpower decreased when specialists were lost to create/coordinate SPI and 
this has increased the workload on the remaining heroes. 

There must be a commitment to the time involved in order to reap the quality 

benefits. | 

Confusion caused by lack of good examples of what documents and procedures 

should be developed. 

Money and time - | have not seen any return on the SPI investment yet 

Lack of available training to be more efficient in our efforts 

Monetary investment has shown minimal improvement - significant number of high- 

salaried employees devoted to SPI. Significant overhead has not shown ROI - too 

much $$, too little improvement 

SEPG that existed early on was not a help - their attitude negatively impacted SPI 

Impression that projects served FSA staff and through them the FSO instead of the 

other way around 

FSO preaches SPi - no support given to focal SPI efforts 

Amount of time spent on activities, juggling priorities (SPI versus customer 

satisfaction) 

Don't know 

Impact on morale caused by the half-hearted implementation 

Generated many forms, steps and procedures which add a lot of time to completing 

the work. 

Adjustments of overhead 

Functional community not receptive to SPI 

Customer complaints 

Tremendous overhead costs - affects productivity 

A proliferation of paperwork and requirements without any clear goals or direction 

Time involved to make even small changes 

Time spent seemed excessive - archival of documentation is excessive - find another 

way of providing evidence 
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Making changes to processes after an AIS placed in production caused some 
confusion & duplicate efforts 

Morale problems - insufficient information received at lower levels - people think this 
involves major changes in how they work, but this isn’t true for everyone 

Takes longer to get the job done 


Sometimes, especially for simple changes, SMS slows the completion of work tasks — . . 


too much 

Increased amount of time it takes 

Requested changes can no longer be accomplished quickly - spend much more time 
documenting and waiting for other people to finish their steps 

Increased time allotted for release 

It takes much longer to make changes in sohware and minor improvements are not 
made 

Added paperwork that needs to be charged to a customer who already doesn’t have 
enough money to get job done 

Significant amount of time expended in reviews and documentation 

Amount of upfront time expended to implement SMS and CMM 

No interaction to determine what the processes are and what doesn’t work or needs 
improvement | 

Confusion as to what projects to apply SPI/SMS standards 

Documentation was distributed and expected to be followed, but no formal training 
was held 

Extra paperwork 

Many SCEs have required additional funding to complete SPI activities 

Lack of “proactive” support from senior management 

Morale 

The amount of documentation er to satisfy the ssauienleiis 

Expensive to implement - costly record keeping 

Customer has claimed slower turnarounds for impacts and releases . 
The layers of extra work and bureaucratic red tape that are necessary to meet SPI 
requirements often seem to be a make-work waste of time, especially for small 
changes/SCRs . 

The reduction on production because of the time spent on doing documentation in 
support of SPI efforts - SPI requires 2 or 3 times the amount of time spent on 
documenting each step or phase - more attention on meeting SPI level than on our 
job 

Terrible cost - too much documentation 

Time required 

Lateness in getting procedures on how to as tasks - with no initial direction, much 
was done without guidance 

Time/$$ to develop all processes/procedures - could have used a Level 2 AIS’s stuff 
and gotten there quicker with less $$ - share stuff - director needs to realize SPI. 
takes time and shouldn't be rushed or date-driven 

Customer/functional staff not willing to get it done “now” - moving train and getting it 
painted syndrome 

Effort to follow SMS as the methodology for reaching CMM Level 2 

Cost and time 

Lip service by senior management - suspension of staff functions indicates non- 
support of SPI effort - no management support - too many lies - our nee are 
dropping like flies - they just give up 

Overburden on AIS staff - unrealistic or uncompromising dates and goals were set 
without input from the AIS - we are understaffed in some areas, and this makes it 
worse. 
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Some “game-playing” on the part of managers who focus on appearance of 
improvement rather than real improvement. 

Additional time to develop various plans and products. 

The initial impact was the vast quantity of paper generated, however, as we progress, 
that should level! out. 

Initial perception was of “program” to be forced on projects. 

Feeling that the reviews and comments associated have negatively affected the 
teamwork attitude the system developers need. 

Time required to fill out forms, develop/maintain documentation reduces the amount 
of work that can get done from one update release to another even though the “old” 
way might have resulted in a 10% rework and re-release. 

Delay of a Release was caused by the preparation of the SCE. 

The additional meetings and paperwork are very time-consuming and lengthens the 
time to complete work. 

Increased response time for handling emergencies caused by need to follow 
procedure. | 

Upkeep of the Business Handbook is added workload. 

Good initiatives start out but without proper funding/value are quickly overcome by 
events. | 

More paperwork. 

Very time-consuming - cost/benefit still undetermined. Customers leery. 


6. Please rate the training you received on SP! Initiatives: 


SEPG AIS Support 
. FSA/DSE Total _ _§_ Managers Members Practitioners Division Other 
FSACL 3.56 3.67 3.5 3.6 3.5 
FSACO 3.43 3.67 4 3 4 - 
FSADE/ 3.00 3.18 1.6 3.13 2.83 
DSE-MP 1 DK/NA 
FSAIN 3.14 3.27 . 2 3.38 2.67 2.9 
1 DK/NA 1 DK/NA 1 DK/NA 3 DK/NA 
FSAKC 3.54 3.57 3.5 3.68 3.57 2.33 
: 3 DK/NA 1 DK/NA 
FSAPE 3.75 3.75 3.75 - 3.50 4.50 
| 1 DK/NA 
Comments: 


SPI training was at appropriate time/level 

More specific training is needed (how the AIS will specifically perform the 
processes/procedures versus the what in the SMS; how to implement the SMS/CMM 
(perhaps from the SEPG); how to translate the CMM/SMS into tangible mechanisms 
(checklists, forms, etc.) 

“Shadow consulting,” change agent training and more descriptions of specific 
benefits for each KPA versus general benefits of SPI were needed. 

Customer at the site level needed “education” on SPI benefits - both generally and by 
KPA. 

Senior managers (division chiefs and directors) could have benefited from training 
related to implementing change in organizations 

Received a lot of training that gave a lot of information, but very little insight. Could 
have been better. 

Training was received on each KPA. This training was provided by FSO. | 
personally received some excellent training from an outside vendor, FASTRAK for 
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CM. Perhaps some sharing among AlSs or visits to see the process in action would 
have been helpful. There has been no training since approximately 2 years ago. 
Training was received 3 years ago. Nothing since. 

Training provided was okay. The problem was that due to inadequate up-front 
planning, the training needed by some was not provided until much too late, e.g., 
TWG members working on process definition. 

Training received varies greatly and not necessarily at the working level. 

Outside vendor training has been excellent. 

Overall in-house coupled with vendor training proved very beneficial. 

The training received was very complete. It was extremely helpful in providing a 
thorough comprehension of what was required for each KPA. 

Training provided by SE! and other vendors was extremely beneficial. Training 
provided by the FSO was weak. 

Some good/some bad to average in the middle. For the most part, internally 
developed training was good/FSO-developed training was bad. 

Working primarily on Requirements Management - requested but could never get - 
formal training on this KPA - feel it caused a lot of wasted time. 

initial overview training was dull, dry, boring. Didn’t like being read to from a book 
that | had in front of me. Learned more by being involved with a local working group 
and the later training sessions given by the Change Agents and other groups. 

It took several sessions to get it all straight in mind, but | think I finally got there. 
Explaining evaluation system helped tremendously. 

Need more whys and theory versus reading CMM/SMS - Gini provided good review 
of SMS/CMM 

STC good as a starting point, but flat overall - could have been more insightful and 
less text - vendor training very good - should have KPA experts within FSA facilitate 
training 

Vague information - sessions would neually degenerate into complaining about the 
proponent agencies 

Very little training was available - no money in budget 
Several requests made for vendor training in CM and SQA, but ail denied due to lack 
of funds. 

Had to obtain own - STC training very poor 

Training from Learning Tree emphasized testing - training from FSAIN staff was 
simply a reiteration of CMM/SMS - my time better spent in reading primary source - 
training from STC (ENSCO) was very good | 


-CMIS class done well, however, without specific examples of using it in real life, | 


don’t see what to do with it 

Still waiting 

Never received training 

| have not received any training for SPI 

| have been trained elsewhere, no training here 

When you don’t get the material, you cannot learn as much 

We have some very talented people in our organization 

Am not asking for more - | learn by putting down on paper in words | understand, 
clarifying and freedom to study - my. learning i is slow but sure --it is not good for in- 
house training : 

All theoretical, not practical - learned | in class - not used in work place = soon 
forgotten 

Occasionally too detailed and long 

Continuous training or awareness sessions are very important and should be done 
more frequently 

Training received at various times, but nothing implemented yet 

| like the SPI demo on software 
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Much redundancy bordering on overkill, but the SMS process was well explained 
These topics need to be scrutinized to make more understandable 
Many handouts were given, and several classes covered what was needed to 


~ achieve Level! 2 


eo @@e@e0ese@3ef: 
Zee 


After given the training, don’t use for months, if ever 

Don't think | received any formal training, and any informal training was not tagged as 
SPI 

Could not attend class 

Need training for all contract personnel 

Need specific training in the assigned AIS 

Needed more CMM training up front 

Extremely dry and boring usually 

Unable to rate for no basis for evaluation 

Need more 

Training given and then expected to get it done right away - How do you roll over 
your environment when you are always in a 50 hour work week in 40 hours before 
SP! 

Training | received was totally inadequate - training needs to focus on system specific 
rather than organizationally 

STC staff has been dedicated and committed to delivering quality training - tough 
mission : 

Why cancel the STC training group? 

While my training was good, we are having difficulty getting training for CM and QA 
personnel as well as other SPI training for SCE 

The training provided a good understanding of the SP! program and evaluation 
process. re. | 
Some areas: sizing, tracking, oversight, seemed shallow. 
Received valuable extra training from SEI due to SEPG and SCE evaluator role. 
FSO training was sufficient, but SP! conferences were exceptional. 

Training was ample and conducted by personne! very knowledgeable of SPI. 

Have only received in-house training. 

Limited access to within Activity Level 2 systems documentation for guidance and to 
knowledgeable personnel for documentation development. We had to start from 
scratch even though our Activity had a Level 2 System. 

We only had it before the CMM audit. 

| feel that the training | have received on SPI has been beneficial. We also have a 
SPI bulletin board. Local training was very good. 


7. Please rate the SEI support you received on SPI Initiatives: 


SEPG AIS Support 
FSA/DSE Total Managers Members Practitioners Division Other 
FSACL 3.64 3.67 4 3.25 3.67 
1 DK/NA 1 DK/NA 
FSACO 3.46 © 4 3.4 © 4 - 
2 DK/NA 1 DK/NA 
FSADE/ 2.91 3 2.5 3.03 2.4 
DSE-MP 8 DK/NA 
FSAIN 3.05 3.67 3. 3.06 2.83 2.56 
3 DK/NA 1 DK/NA 8 DK/NA 3 DK/NA 4 DK/NA 
FSAKC 3.45 3.17 3.25 3.95 2.57 2.67 
1 DK/NA 6 DK/NA 1 DK/NA 


ZZ 


FSAPE 3.67 4.0 3.67 3.2 4.0 
1 DK/NA 1 DK/NA 


Comments: 
e SE! was very helpful 
_@ Not much contact by AIS personnel with SEI - some by ccmail 

e Support at the SCEs was excellent 

e More direct, hands-on support from SEI at the site level (shadow consulting, etc.) 
would have been beneficial. The lesson here may be that the SEPG and site needs 
more consultation support to facilitate quicker and easier SPI implementation. This 
doesn’t have to be full-time, but periodic consultation at critical stages. 

e Did not have much interaction with the SEI personnel. | know that some SEI reps 
came here to help us with reaching Level 2 in the area of PM. However, | cannot rate 
the support given. 

e SEl has been very cooperative with lending sipseit 

e Very beneficial when the opportunity arises that one-on-one communication is 
possible. At other times, when an issue is raised with hopes of comments from SE}, 
that is a little tougher to accomplish. 

e SEl’s support was not extremely visible to me. | would have to say that the majority 
of our support came either from the FSO or FSA level. 

e What we were told we had to do to become Level 2 in PM did not mesh with what 
other AlSs were doing who were Level 2. Do not agree with SEI opinion expressed 
that “you cannot do too much ...” - in this case PM, but seems to be a common 
theme. | 

e The shadow consulting we received was invaluable. 

e . Rozum was knowledgeable and ready to assist when needed. 

» SEIi who? NO, actually, it was good at times, but often resulted in confusion from the ~ 
mixed messages that were received. 

Support has always been very helpful. 

| have never been clear on the scope of the SEI agreement. ‘Would they have helped 
me on my initiatives if | had asked them? | felt SEI was very short on concrete advice 
- they were more like a stereo-typical therapist, answering “well, what do you think 
you should do?” to every question. We’d have been better off on our own. 

e My position as practitioner didn’t put me into contact with SEI. 

e Don't know if any support received at project level 

Available to answer questions and analyze solutions - early on, wished more direct 
answers 

None, other than having a copy of the CMM 

Never had direct contact with SEI 

Hardly ever here - never got direct answers 

| have not received any SEI support 

Not aware of SEI involvement 

| have not received any SE! support 

| believe some SE! support could be very helpful to the effort 

Learned from others in the office and trial and error 

Written materials only, no explanation 

| have had no contact with SEI 

Co-workers and branch chiefs are supporting effort without much senior management 
help 

Too academic 

Not aware of what they have done 

e {personally know of no SEI actions although they would have taken place at a 
different level | 

e Questions were answered in a timely manner 
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Not much 

Have not read CMM 

Unknown 

More SPI support would be helpful - they should monitor our practices more closely 
and give us guidance on what improvements need to be made 

They were afraid (so it appeared) to give yes or no answers when it came to 
procedural questions 

What does SEI stand for? 

| have no contact at this level 

Much has been left to the users of the procedures to guess at 

SEI members were learning processes and were inflexible in implementation of the 
SMS 

They've been at our beck and call and have always responded/supported us 

Too many flip-flops on how we do our business 

We haven't had any yet on our project 

SEI support helped to clarify terminology and facilitate the crosswalk from current 
practices to the CMM. 

SEI provided continuous on-site and off-site support that was a significant factor in 
improving software engineering practices to Level 2. 

They were very helpful in zeroing in on what needed to be done for an assessment. 
SEI sent a representative to do an evaluation prior to the ae His input proved to 
be critical in many areas. 

We had one sit-down session with an outside SE/ person in the 8-12 month process. 


8. Please rate the FSO support activities to SPI Initiatives (These activities include 
training, FSO SPI conferences, SMS workshops, and process definition.): 


SEPG AlS 


Support 
FSA/DSE Total Managers Members Practitioners Division Other 
FSACL 3 3 3.2 2.75 
FSACO 3.73 3.33 3 2.83 1 DK/NA - 
2 DK/NA 
FSADE/ 2.75 2.4 2.2 2.9 2.8 
DSE-MP : 3 DK/NA 
FSAIN 2.82 3.38 2. 2.96 2.43 2.4 
4 DK/NA 1 DK/NA 7 DK/NA 2 DK/NA 3 DK/NA — 
FSAKC 3.12 2.67 2.5 3.48 2.8 2.67 
1 DK/NA 5 DK/NA 2 DK/NA 1 DK/NA 
FSAPE 3.0 3.0 4.0 2.25 4.0 
2 DK/NA © 2 DK/NA 2 DK/NA 
Comments 


As an organization, it seems we focused too much on the SMS and should have 
focused on the CMM sooner. 
There seemed to be a communication gap in getting information from the various 
FSO SPI conferences, etc., to them. 
The value of the SPI conferences was questioned when the main agenda was status 
reports versus sharing solutions, setting direction, resolving issues. The VTC is an 
excellent format for status reports. However, face-to-face is more beneficial when 
synergy is required (i.e., setting direction, alternative approaches, and resolving 
issues/obstacles). 
Although | only experienced some training plus one SPI conference, | know that there 
was plenty of support activities for SPI by FSO. 
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Process definition activities of TWGs did not provide for equal input by all participants 
when they were initially advertised as such. FSO TWG participants did not contribute 
meaningfully to TWG process definition activities. 

Training provided was fine. Support and process definition were very sketchy at 
times and not very conscientious. 

| would have to say that support shown by the FSO was extremely high considering, 
the training dollars allocated. 

Poor direction and metrics and FPA leads to useless work and data that in some 
cases we know is meaningless. Important one day and not the next. Adds more 
overhead. . 

Only the training that was contracted was worthwhile. Training on the tools and on 
the individual KPAs was poor. 

FSO needs to take a more team approach, not dictatorial. 

The process definition efforts were marginal at best. Processes were dictated rather 
than created with the input of all FSAs. 

Reading slides and the SMS to software engineers is insulting. SPI eo ren ee 
shared little useful information for their costs. 

Felt we were being “force fed” processes/tools. 

Other than selected members of the Steering Group, | really felt we were pretty much 
on our own. 

Not involved 

Gini did great - miss SEPG newsletters - Anita did great job on maintaining and 
soliciting feedback on SMS updates 

Never attended any of these except training 

Never attended any of these and did not know they were available 

Would have done better if each FSA pursued Level 2 on own 

Not sure that | have been involved with these FSO activities 

| have not received any training or support of this nature 

Look at the committees charged with developing SPI materials 

No money for training or attendance 

| have had unit test, SQA, and PM - one exceptionally good contractor instructor - 
keep this man - have not been to conferences or workshops 

Non-existent 

Definitely went extra mile on training - we had many individual training courses and 
special class prior to rating - class was tailored for our system and was very helpful 
Really hard to judge because of the difficulty of the tasks 

Haven't attended any for a long time’ 

Except for basic/overview training, who gets to attend/participate in their activities 
Support is good but could be better if classes were given more often and were not so 
long - ideal is 1.5 hours 

My experience here is limited to formal training that was funded by the FSO 

SMS Is rarely talked about until an impending meeting, then it is the most important 
issues we have 

Classes and booklets were given to everyone in the division 

At first, FSO was active - now dropped out of sight 

| have not been involved © 

Not directly involved 

The training needs to be geared around system specific processing 

What FSO activities? None to speak of - when they did come in, it was as dictators 
and giving wrong information - FSO lost credibility in many people’s minds 
Over-definition of some processes, not enough on others (policies for Web, MTMO,. 
oracle, etc.) would prefer leadership and strategic planning to nicromanaging 

Good at first, but seems to be losing steam. 
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e These were very helpful to SEPG members, but not sure how valuable to 
practitioners. 

e FSO provided leadership and budget support for the SPI program. Without this 
support, very little would have been done. 

e Training was mostly to explain terminology and “read” the KPAs, not “how to” on 
development of policies/procedures. 








9. Please rate the tools provided by FSO for SPI Initiatives (Tools include LRS, CMIS, and 
SMS Survey Tool.): 


SEPG AIS Support 
FSA/DSE Total Managers Members Practitioners Division Other 
FSACL 2.75 2.67 3 3 2.25 
FSACO 2.79 3 3 2.57 3 - 
FSADE/ 2.16 2.09 1.83 | 2.24 2 
DSE-MP 1 DK/NA 
FSAIN 2.64 2.44 2; 2.96 2.25 2.42 
3 DK/NA . 1DK/NNA . 3DK/NA 1 DK/NA 1 DK/NA 
FSAKC 2.70 1.86 1.75 3.07 3.00 2.00 
1 DK/NA 2 DK/NA 1 DK/NA 
FSAPE 3.41 3.25 4.25 3.00 3.33 
Comments 
e FSO SPI Tools help get the job done but are not user friendly and can be 
cumbersome. 


e Overall approach to émploying support tools seemed disjointed. The tools with the’ 
least support are CMIS and the SMS Survey Tool. It appears that what people want 
is an integrated tool set for each of the functions. For instance: 

a) Asingle integrated too! set that performs CM control/migration and status 
accounting, and Cl inventory. CMIS does only 2 of these. 
b) A single integrated tool set that helps track hours and other project management | 
activities. 

A possible alternative is to purchase COTS that are somewhat tailorable. 

e Need to purchase standard CM, test, and project estimation tools. 

e Concerned that tools being used have not all implemented and reached CMM Level 
2. CMIS could not provide a CM plan, for example. 

e LRS and CMIS are good tools to use. The Survey Too! was basically useless a year 
ago when it was needed. 

-@ {donotuse CMIS. LRS is fine. SMS survey tool | cannot recall what this was. 
Again, the tools duplicate processes like estimating and PM activities and are not 
necessarily designed for the AIS. 

e LRS is very prone to breakdown. The SMS survey tool became less useful as the 
focus shifted from SMS to the CMM compliance. 

e Sometimes the release of the tools to accomplish SPI initiatives is difficult to 

' determine. 

e SMS Survey Tool is ; the most beneficial of these tools; however, the too! would be 
more useful if it would be based entirely on the CMM. 

e Tools are very poor. If we are to proceed with this effort, we are definitely in need of 
additional automated tools. We cannot be expected to support a Level 2 or 3 
organization with archaic tracking devices. 

e AlSs expected to adopt to tools developed for AlSs in different environments (i.e., 
CMIS). Trying to force round pegs into square holes. Attempts to use LRS for 
metrics in ways it is inadequate for. 
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LRS is the best of the bunch. CMIS was a bad decision based on a lack of 
knowledge of what was needed. It was proven not to be the universal tool as was 
advertised. The SMS Survey Tool provided no benefit. 

CMIS = poor performance/not adaptable to large systems. 

These tools are poorly integrated, difficult to use, and are an insult to us as a 
software development organization. 

They all have that awkward, home-grown feel. Probably would have been better to 
find tools that suit the needs rather than enforcing tools to fit needs that were never 
envisioned when they were developed. 

LRS is probably the best of a bad bunch. Has anyone besides me been appalled at 
the amount of money this place has spent on CMIS? We should have bought COTS, 
then built the SCM function around it, instead of having everyone keep trying to 
change the too! to match existing practices. 

CMIS is old and outdated. Written by individuals for specific site and everyone is 
expected to use. CCB process has broken down. LRS is not designed for a large 
organization. Lots of problems with software. 

LRS and CMIS should talk to each other and the SMS survey tool was cumbersome 
and an exercise in futility of status reporting for no purpose other than HQ had a tool! 
It appeared CMIS tool was picked without study of methodologies at sites. 

PANAPT is best. 

It seems like a lot to fill out to keep track of what you are doing. 

LRS for entering time and don’t use CMIS 

Could have used more definition on overhead categories - SMS survey good to 
maintain checklist when first implementing - could not load new version as too big - 
CMIS helpful but need tools for TDR status, version control, sizing, library controls 


' CMIS needs something like a tiger team, not endless meetings, in order to implement 


Input correct time into LRS, then maybe we can use it as a tool to measure progress 
- CMIS needs a library system to be effective | 

Only training received was on CMIS 

LRS times not entered correctly to provide accuracy - CMIS oy as good as data - 
better if linked to library system/LRS 

CMIS very good - LRS sufficient - survey tool inadequate 

CMIS is nice report tracking tool but just adds duplicate effort and does not control 
software movement and versions 

Only involved with CMIS to limited extent - it controls documentation and approvals 
but does nothing for physical configuration management 

Have only taken CMIS training and not using it as part of my duties at this time 
What do the tools names have to do with SPI? We need to implement Designer 
2000 as a process tool 

LRS is good - No CMIS training - SMS: next version should be much shorter 
Multiple data entry is insufficient 

We have LRS - we use OSEDT, not CMIS 

LRS: eeny, meeny, miny, mo, what should | charge the customer for today? CMIS: 
“user friendly” not an attribute of this exercise in futility. SMS Survey Tool: Have not 
seen nor do | know what it is/does. 

LRS and CMIS are poor - data collected may be useful at FSO but are of no worth to 
me - both impractical. SMS-OK 

New LRS pending - has been delayed again. CMIS is not functioning yet. 

Never see anything happen because of surveys. Not using CMIS yet. LRS is OK but 
not very useful. Need report that will give summary total of how many hours were 
against a specific SCR. 

These tools were very much needed and make our jobs easier 

CMIS could be improved ) 

OSEDT is a great too! - LRS seems to be very good also. 
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On-line (LAN) tools are used by everyone in the division - LRS, CMIS, Word, Excel, 
ccmail are a few tools used 

They helped 

CMIS very good 

SMS and LRS are functional for what they are designed to do 

We've had CMIS training, but haven’t started to use it yet 

LRS is hard to use - New CMIS is needed - Training 

LRS and CMIS proved invaluable. The survey tool provided little value 

LRS is used, and it is on the network. 

Does not provide what we need at the developer's level 

CMIS does not add value to our processes but makes them more cumbersome 
because of additional procedures 

Couldn’t use CMIS due to inadequate environment. LRS cumbersome and needs to 
be more user-friendly. Data not always accurate. 

Still trying to get there. 

We were tasked to perform mission with tool incapable of meeting requirements 
LRS is a joke - no one enters correct information for it to be of any value. CMIS is 
okay. SMS survey tool is a joke. | 

You must be joking - not user friendly - not compatible - no interface - redundant 
LRS has been co-opted by business management people and is virtually useless as 
a source for software metrics. 

Both LRS and CMIS were significant factors in our achieving Level 2. These tools 
provided cost and effort data as well as manage and teach the requirements. 

All three of these tools were extremely useful, if used, to assist in managing and 
tracking. | 
Tools are valuable but there needed to be more standardization across FSO for 
usage. | | 

CMIS vital for CM portion. 

They do not provide for cross-walking as each is structured so differently. _ 

LRS and CMIS were part of our practice before SPI came along. | don’t know what 
the SMS Survey Tool is. - 

Familiar with tools, but not with FSO support. 

Most tools used are standalone. A greater improvement to the software process 
would be a more integrated unit. There is no link between CMIS and LRS. There is 
no link for CMIS and our Version Control System. Each piece acts alone. 

LRS and CMIS are Very Good. SMS is Inadequate. 


10. Please rate the local SEPG support you received on SPI Initiatives: 


FSACL 
FSACO 
FSADE/ 
DSE-MP 
FSAIN 


-FSAKC 


FSAPE 


SEPG AIS Support 
FSA/DSE Total Managers Members Practitioners Division Other 
3.94 4 4.25 3.8 3.75 
3.85 4.33 4.5 3.43 4 - 
2.89 2.81 , 2.8 3.0 2.33 
2 DK/NA 
2.96 3.4 2.0 3.24 2.14 2.5 
2 DK/NA 1 DK/NA 4 DK/NA 1 DK/NA 3 DK/NA 
4.07 4.29 4.00 4.2 3.71 3.33 
3 DK/NA 3 DK/NA 1 DK/NA 
4.00 3.75 5.00 4.0 4.0 
3DK/NA | 1 DK/NA 
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11. Comments on local SEPG support: 


for 


SEPG was helpful in implementing SPI initiatives 

Wide range - from ‘absolutely no value’ to ‘only reason the organization made any 
progress’ 

Need to ensure the SEPG is knowledgeable, experienced and can give examples of 
mechanisms to the AlSs. The “standard” mechanisms could come from either the 
site or FSO level, but people need to have samples of mechanisms to translate the 
CMM into workable processes. The lessons here may be: 


a) Formulating and implementing a training/education agenda for SEPG 
b) Network frequently with other organizations implementing SPI, in order to share 


approach, lessons learned, and sample. 


c) Obtain consultants periodically at key points in implementing the SPI program 


both the FSO-HQ and local site levels. | 

CM and SQA provide excellent support for AIS and for SEPG members. 

All local SEPG members were very helpful and informative. They were always willing 
to help research the answers to questions. 

They are very helpful. 

In some areas, the support was exceptional, but others it was marginal. 

The SEPG is very well run and members are knowledgeable about their areas of 
expertise. If asked, they've provided valuable information to all. 

SEPG has assisted in crosswalks review of implementation plans and assessments. 
Again, this is somewhat reflective of the FSO support received. 

| believe that the SEPG group was instrumental as far as serving as a liaison 
between the FSO and the FSA. 

Sometimes overzealous in requiring artifacts that have no value to the AIS. Apply | 
procedure for the sake of procedure regardless of any benefit. 

| don’t think that the SEPG has ever realized that a group is as powerful as people let 
itbe. For example, if they’d have sent out a few directives “from now on, everyone 
must ...” instead of trying to get everyone to embrace their philosophy, they’d 
probably gotten a lot of stuff implemented. As it is. people feel they had a choice and 
some chose no. 

The local SEPG’s support was the key to the successful SP! implementation at 
FSAKC. 

Our SEPG was ALWAYS thers to solve problems and answer questions. 

Very hard-working and conscientious. 

Without them, there is no Level 2. . 

Can't say enough good things about our SEPG Team. | 
It seemed to me the SEPG assisted SMD personnel. | didn’t realize until later this 
was actually a SEPG function. Also, | am not aware of all the efforts of the SEPG, 
only those that directly involved me. 

| get SPI news almost every month and Word of Week. SEPG does excellent work. 
Make better organization for designers, programmers, and testers. Saves US | 
Government a lot of money. 

Our SEPG Team has made a valiant effort, but had only received a lukewarm 
welcome. . 

Could be more proactive, leading the way, bringing the practitioner needs and ways 
to meet them to management, rather than limiting their efforts to what they feel they 
can talk management into. 

Once AIS is educated, SEPG should act as consultants, not implementation project 
managers. SEPG needs to attend conferences and spread new ideas for support. 

! am not familiar with the SEPG. 

Don’t have an SEPG anymore. 


This group needs more PR to let the rest of the world know who/what they are/do. 
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e No visible SEPG. 

e Early on, they treated software engineers as if inferior. Took credit for work they did 
not do. 

e |have not received any local SEPG support. 

e Support is lacking. 

_@. None. 

e Not aware of SEPG support. Just started working with SPVKPA teams. 

e |am too far down the chain of command to have contact. 

e What SEPG? 

e consider them overhead. 

e Occasionally spread too thin and occasionally impractical with uncooperative attitude. 

e Always ready to answer questions and to do what they can to help us understand 
and set goals for SPI. 

e Any questions are answered in this area. 

e Not much too. 

e Unknown. 

e Our division reps keep us very well informed about SPI activities. 

e | don’t know what SEPG is. 

e More SEPG support will be helpful. They should closely monitor our ir practices and 
give us guidance on what improvements need to be made. 

e Very supportive. 

e Not properly staffed. 

e What does SEPG stand for? 

e Nocontact 

e There has been little or no support shown 

e Provided support when requested 

e Good effort but failed 

e Never supportive. Always dictatorial aad combative 

e No SEPG - inadequate when we did have one. The process has become a total joke 


in our FSA. The other FSAs are doing it better. Ours has done a complete 
turnaround on our vision and methods. 
e SEPG is currently understaffed and undertrained through no fault of their own. Goes 
back to management (FSO/FSA) support. 
e Loca! SEPG provided support as requested to project SPI staff. 
e | thought the support was quite good, once an AIS was receptive to it. 
e All members were very knowledgeable and cooperative. They contin ued to go out of their 
way to facilitate a project’s improvement. 
‘ = ©We have a very knowledgeable and helpful SEPG. They have assisted me and provided 
guidance concerning procedures. 


12. Please rate the effectiveness of the SCE process: 


SEPG AIS Support 
FSA/DSE Total Managers Members Practitioners Division Other 
FSACL 3.13 2.67 3.5 3 . 3.25 
FSACO 3.43 3.67 3.5 3.14 4 - 
FSADE/ 3.45 3.8 3.5 3.41 2.67 
DSE-MP 3 DK/NA 
FSAIN 3.91 | 4.17 3.5 4.2 3.67 2.57 
3 DK/NA 6 DK/NA 
FSAKC 3.48 2.86 4.0 3.70 3.43 2.67 
5 DK/NA 1 DK/NA 
FSAPE 3.94 3.5 4.25 4.00 4.00 
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1 DK/NA 


Comments: 


Most comments centered on the accuracy of the assessments and the help provided 
by the final findings reports. The assessments were viewed as professional, fair, and 
accurate even when the findings/results did not result in a CMM Level 2 rating. 
Could have been better and more consistent if the process had been documented. 
The SCE team members were very professional. It’s believed that the SCE ploseee 
was not uniformly applied between FSAs. 

| do not feel the SCE criteria have been applied consistently. Credibility is lacking. 
Never been involved in actual SCE 

What is an SCE? 

Do not know what “SCE” is 

What’s the SCE process? 

What is SCE? 

Not done yet 


13. Please rate how well the SCE findings helped in action planning for future 


improvements: 
| SEPG AIS Support 
FSA/DSE__Total__§_ Managers Members __ Practitioners Division Other 
FSACL 3.56 3 4 3.6 3.5 
FSACO 3.31 4.33 3.5 | 3.5 3.5 - 
, 1 DK/NA 
FSADE/ 3.64 3.91 3.6 - _ 3.59 3.33 
DSE-MP 7 DK/NA | | 
FSAIN 3.85 4.17 3.5 4.11: 3.75 2.43 
| 5 DK/NA 6 DK/NA 
FSAKC 3.47 3.00 3.75 3.73 3.29 2.67 
| 6 DK/NA 1 DK/NA 
FSAPE 3.81 3.25 4.0 3.8 4.33 
1 DK/NA 


Comments: 


Most comments centered on the accuracy of the assessments and the help provided 
by the final findings reports. The assessments were viewed as professional, fair, and 
accurate even when the findings/results did not result in a CMM Level 2 rating. 
Answer relates only to improvements relating to achieving Level 2. 

Not evaluated yet 

Identified weak areas - 

Don’t know what “SCE” is 

Not done yet 


14. Comments on assessments and related activities: 


SCE tied everything together and helped identify areas needing fapravenient 

SCE findings were too vague 

Most comments centered on the accuracy of the assessments and the help provided 
by the final findings reports. The assessments were viewed as professional, fair, and 
accurate even when the findings/results did not result in a CMM Level 2 rating. 

The assessments were helpful in identifying weak areas in our processing. The IP 
was helpful in getting an insight of how the workers viewed the process. 
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The lessons learned were helpful for the next projects. SCE process should have 


_ been documented to provide more consistent evaluations. The success/failure 


perception was overemphasized. 

The SCE pointed out the same areas of improvement which we knew were weak. | 
think the SCE should be done by outside people, outside the FSA, not by peers at 
other centers. 

Current teams are working well. 

The assessments are a good measure of implementation progress. 

Some inconsistency among the team members possibly due to misunderstanding of 
the actual organization’s process or possible pressure from other team members to 
rate otherwise. : 

The assessments were very helpful in evaluating our progress and in preparation for 
the SCE. 

Credibility is not what it should be. 

The SCE findings were very accurate and quite useful in developing programs to 
change processes. __ 

It was extremely frustrating to go through the process, for the extended visit and have 
the team decide they could not give us a rating. Not apparent guidance when a 
stalemate occurs between members of an evaluation team. 

While painful at times, the SCEs drove home the CMM concepts and helped pull the 
FSA together. Because the results were focused at Level 2, the findings only 
addressed small improvements. 

Don't think they were done fairly across the board. Some systems appeared to be 
under tighter scrutiny than others. 

The “pass or die” attitude wasn’t the fault of the SCE process.. There were hard 
feelings about the SCEs. The concept of having in-house assessors was good, but | 
in practice it was a losing situation - unless the site passed - then you could be pals. 
How many sites followed their “recommendations for improvement” to implementation 
once they got Level 2? Findings were good, but | doubt the level of follow-through. 
Interpretation of some CMM elements are vague and leaves too much discretion to 
the assessment team members. 

Need to have time to operate at Level 2 and iron out procedures/processes before 
proceeding to Level 3. 

They take too much time and resources if they're only to be used to establish a 
grade. They'd be very worthwhile if they were used to focus and plan improvement 
efforts. 

Still waiting for this assessment. Questionnaire should have been postponed 60 
days. | 

Can feel political and competitive at it’s worst. Can feel worthwhile and instructive at 
its best. 

The findings | read were more about the SMS, not CMM. 

Headed in right direction, but still a lot to do. 

Dates set for an AIS evaluation are often set outside the AIS and do not consider 
need to implement, train, improve and institutionalize the processes for the AIS. 

The extra forms, steps, and procedures should add much cost and waste to work 
Very good for efforts to be applied 

None of constructive value : 

These are enlightening and beneficial - Determine the extent of the SPI 
understanding | 


_ Assessments of immeasurable value 


SCE helped me identify where | was lacking knowledge of the process, so | could 
work on improving my awareness in these areas 

The first assessment was a little confusing as to what was needed. The second one 
was much better. 
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Weaknesses were pointed out and explained. 

The questions sometimes seem vague and obscure. Maybe ih need more 

explanation, or to use fewer unknown/forgotten acronyms 

No SCE yet 

Some questions on SCE were value - more probably misinterpreted 

Pointed out weaknesses for you to improve upon before next assessment 

Question qualification and experience of assessors - Prefer to contract out to an 

organization experienced in assessments 

Too many of the assessors had hidden agendas. Revenge, getting even, etc. 

e Assessments by trained personnel are fair and objective. Much goes into the 
process. Strengthens, and weaknesses are clearly defined. Other information 
provided for future improvement. 

e Assessment was conducted in an impartial and professional manner. 

e Assessment was a stressful time. 

Assessments helped set direction and identify soft areas in our software engineering 

processes. 

Provided areas where improvements could be made. 

Very thorough. 

Assessment was very fair. 

They have been conducted in a professional manner. It is a very good practice to 

interview workers separate from managers. 


15. Please rate the usefulness of the System Modification Scenario as a process 
definition: 


| ~SEPG © AIS © Support 
FSA/DSE Total Managers Members _- Practitioners Division Other 
FSACL 2.56 2 2.5 2.8 2.75 
FSACO 2.92 2.67 3 2.86 3.5 - 
FSADE/ 2.56 1.9 1.67 2.83 2.67 

DSE-MP | 1 DK/NA . 
FSAIN 3.07 3.17 2. 3.17 2.88 2.91 
1 DK/NA 1 DK/NA 1 DK/NA 2 DK/NA 
FSAKC 3.16 2.17 -2.00 3.58 3.43 2.67 
1 DK/NA 2 4 DK/NA | 1 DK/NA 
FSAPE _~—_si32.21 3.0 3.25 4.33 2.33 . 
| : 3 DK/NA 


Comments: 

e SMS is too detailed 

e SMS is oriented to another site’s process and doesn’t impact daily work 

e SMS is too bulky, complicated, difficult to read and follow. It should have more 
graphics. Templates for various mechanisms (SDP, SCM Plan, SQA Plan, status 
meeting agenda/minutes, checklists for SQA and/or CM audits, etc.) would have 
been helpful, but not the detailed expansive how-to narrative procedures. It 
appeared to be too detailed where it wasn’t needed (how to set up a meeting) and 
not detailed enough where it was needed (what should be done in a baseline audit, 
what should be the outputs - even a sample of minimum information on a template). 

e Conflicting direction on what had to be implemented in the SMS (i.e., task level 
versus procedure level) was frustrating and time-consuming. Most respondents’ 
comments agreed that task level (the what) is the appropriate level for — 
implementation. Lessons here may be getting more consultants early and at critical 
SPI program milestones at both an FSO-HQ and site level to help guide the direction 
and point out pitfalls to avoid. 
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Sufficient for projects to use as a template to tailor their own process. 

| think the SMS is useful as a process definition for new development. It is not useful 
as a process definition for modification. 

| believe that the SMS is adequate for new development, but for maintenance, it is 
overkill. 

Had the communication and planning by FS senior management been better up 
front, the product, SMS, would have been better. Also, if the various FSA TWG 
members truly had equal input, the product would have been better received. 

The SMS is a good policy guideline, but since the assessments are based on the 
CMM, it is rather redundant or confusing. Procedures are totally tailored in any case; 
therefore, the SMS is generally superfluous. 

Adherence to the SMS resulted in consistent failure. Inadequacies relate directly to 
Survey (question #1). 

Despite the fact that we are one “FSO,” we are many different systems (size, 
platforms, sites) with many different customers. It is very difficult to put this in one 
book. 

| consider the SMS sufficient as a starting point. To some, areas of the SMS were 
misleading - example, estimation. 

It’s fine as an overall FSO policy, but not quite as useful as a software process 
definition tool. ne CMM better defines the necessary goals and should be stressed 
more. 

The SMS is an Sarenialy detailed manual. In the early stages, it was quite confusing 
as to which manual to use in pursuing Level 2 objectives (SMS versus CMM). 

Goes way overboard on creating an inadequate a mount of overhead which does not 
enhance or add value to the system goals. Too rigid in requiring too much. 

. Too hard to follow, repetitive, and inadequate in reaching Level 2. Some parts were 
useful but as a whole the document went into too much detail. 

Some pieces of the SMS were useful in improvement activities once you could find 
them. The format structure and inflexible nature of the document made it unwieldy 
and practically impossible to use. An organizational process must be flexible enough 
for each project to tailor it to its specific needs, customers, etc. 

Not bad to use as a starting point or template, but still required a lot of other work to 
get to Level 2. Too many CMM areas not covered. NOT USER FRIENDLY! 

As with anything written by a lot of people, then compiled together, the document is 
uneven - some places have too much detail - others not enough. It’s better than its 
reputation would suggest, but | find it criminal that FSO ignored the problem that 
implementing the SMS did not make the site CMM Level 2. 

The SMS is difficult to follow, is too procedurally based, and tries to mandate 
standardization. 

Lots of good information, not the easiest document to read. 

The SMS tended to get too far into the day-to-day operations on some areas. This 
appeared to stem from the SMS author FSA’s frame of reference to their local system 
and didn’t seem to take the basics into account such as the difference in size of a 
local or FSO-wide AIS versus service-specific pay related systems. 

Too big, bulky, way too much detail. Outline the intended function and the results 
you desire (CM - version control, etc.) and then step back and let the individual 
activities fulfill these processes according to their specific needs. Audit periodically 
by FSO. 

At first, seemed painful and redundant. After working with it, could see the elegance. 
Good procedures model, but will not get an AIS versed in CMM or to Level 2. 

“PTR” in SMS should be changed to Program Trouble Report to match CMIS 
terminology. 

It needs to be flexible. Not every project is the same. What works for one may not 
be cost-effective for another. 

Haven't worked with this much. 
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FSO/FSA management focused too much on SMS for reaching Level 2. Lost site of 
CMM. Evaluations against CMM. 

Could have developed own procedures faster. 

Too repetitious. Too cut and dried. Leaves no room for tailoring to the AIS. Seems 
_ to put a straight jacket on the CMM. 

| have one . 

Explains overall process, but each project must devote resources to interpret 
document, and train the team. 

A hindrance to higher levels. Not CMM-compliant. Replace with generic SDP that 
addresses process 

Most of it should make us more inefficient. 

Incomplete. Should be tied completely to CMM. 

Why do we not follow the book instead of making up our own version? How does 
CMM fit in? 

Probably would be very good if properly managed and people had some idea of how © 
it worked and it could be explained clearly and sensibly. 

It required a lot of interpretation and translation in order to be useful. 

Too many things at once. Oversold as the cure-all. Didn’t fit well with production 
support or system development. 

SMS is a good tool. Proved to be helpful in understanding CMM. However, CMM is 
what rated against. Differences in interpreting the two documents caused problems. 
CMM is what we must master. 

It is totally relevant to my job. 

Usefulness depends on implementation within the project. If implementation iS poor, 
SMS is seen as a detriment. 


e | think the overall process is a sound concept, but it is difficult to implement within. 





SMS covers what is needed to define, fix, and change our software. It does seem 
wordy but covers the SMS 

lf all facets are adhered to, and the customers, who do not fall under the SMS 
umbrella, understand what we are doing, SMS can and will be an excellent tool. If 
not, it will be an exercise in futility. 

Lacks CMM requirements to meet Level 2. 

Needs to be formatted for use by all involved and contain the information necessary 
to implement, i.e., checklists, timelines, process flows, etc. 

-For our AIS, we are instructed to also follow Military Standard 498. 

Needs to be interpreted for clarity. | 

Found it to be very beneficial as a starting and reference point for standardized 
procedures. 

Good for complex changes, complicated and too involved for most (shorter, simpler) 
SCRs 

This increase paperwork. Slow down processing. Increase possible mistakes 
because of added steps. Increase uncertain for trying to learn so many processes. 
Very good concept. Implementation time is not sufficient to ensure not just 
compliance, but sufficient knowledge of the process is acquired to be successfully 
repeatable 

There is no documentation on how to use the process. There has been little “buy in” 
by the end user of the process. 

Format useful as checklist for tasks/subtasks to perform. Some terminology is 
difficult to understand. 

Doesn’t have CMM stuff in it. Evaluations are based on CMM. SMS won't get you 
there. 

What can be said for something that slows the process to its crawl. 

Version 3 was a better product than Version 4. 

It's awfully big and cumbersome. 


135 





Too many options. 

The DCPS project uses the high level steps identified in the SMS. 

Presentation method not effective. Some very good concepts and practices 
contained therein but needs to be supplemented with graphics and other 
presentation techniques to communicate better. 

It provided good procedural assistance where we were short. Initially, the format was 
an obstacle to usefulness. Much better now. 

The SMS contains all the processes for Level 2, some at more level of detail than 
others. 

The process tends to bog down in minute details that sometimes don't seem to 
pertain to the work at hand. It does, however, allow for exceptions when necessary. 
Too hard to follow, read, implement, and not realistic. 

Too lengthy to use effectively. 


16. Please list any additional comments on Software Process Improvement: 


SPI seems to be an academic exercise for systems phasing out 
Not convinced that SPI is worth the cost in time and resources 
Change agents and sponsors at the local level need training and regular periodic 
consultation support 
The SMS needs to be more streamlined and graphical with templates 
The SPI program process (and the SEPG) should provide more samples of what the 
procedures and plans should contain early on (versus after-the-fact evaluations by 
SEPG and SCE teams - pointing out the shortcomings). 
Develop SPI infrastructure early (PAL; samples; training of sponsors, change agents, 
and development staff; streamline status reporting method and format; identify 
related metrics - usable for site and HQ). 
Work out strategy and tactics so people aren’t overloaded with changes (CMIS, SPI, 
Metrics, FPA, etc.) 
Perform pilots to work out kinks in the process 
FPA as a sizing mechanism is not cost effective for already established AlSs, and 
does not help in comparing estimated to actual during development 
Positive aspects: 

e documented procedures 

e SQA processes and reviews 

e justification for FFPs and schedules 
| think we have gained some bit of control over our processes - however, the cost for 


_ this bit of improvement has been great. Increased time and paperwork have - 


overshadowed the benefits gained and to be gained by SPI. Unfortunately, many of 
the benefits of SPI are masked, even totally hidden under the amount of process and 
documentation required to reach level. 

| was taught and believe that in the face of the inevitable consequences, it is better to 
say nothing than complain. 

As we move toward Level 3, emphasis on the SPI program needs to be renewed. 
Currently, inactive TWGs need to be reactivated and/or other TWGs started. 
SEPG/TWG representatives need an opportunity to rotate out of the duty or to state 
an interest in continuing in the same or a new SPI role. 

Must address cost of the program and as budgets get cut, this will bea bigger issue. 
SPI is a worthwhile initiative but sometimes it seems to take more time than the 
actual work. Perhaps with repetition, the extra time will be mitigated. Currently, it is 
too labor intensive. 

There is a great value in implementing disciplines of CMM, however, an objective 
evaluation of Level 2 implementation including cost effectiveness should be 
conducted before any other systems begin. This is especially true for smaller 
systems where all resources are tight. 
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With current and future budget restraints looming on the horizon, personnel 
maintaining smaller projects feel that with additional SP! requirements levied on them 
may mean the demise of other projects. An overall FSO plan needs to be developed 
for SPI small system implementation guidance. | 

SPI ideas have helped our organization to take the steps needed to improve our 
aged software development activities and procedures. DRAS software development. 
procedures and activities have undergone a complete renovation. | do not believe 
that this level of change could have occurred in such a short time without an effort 
like SPI. 

Most of the objectives of SPI we were already doing in a less formal manner with a 
record of very few production problems. SPI has added elaborate procedures and 
forma! documentation of every interim minor step taken for the sake of proof. We 
have just as many production problems now and we are just as dependent, if not 
more so, on expert individuals. SPI was also forced on us in a timeframe which did 
not allow institutionalizing or developing more automated tools. It was a much more 
painful process that it had to be. In summary, there is a tendency for SPI to become 
self-perpetuating as processes become more complicated, causing more human: 
error and thus requiring more oversight and more procedures. 

Future process improvement efforts need to include the entire organization to 
succeed. The improvements that survived the push for the grade did so because the 
projects were involved in their development. This increased the buy-in and 
acceptance between the projects have a stake in the improvements. Most of the 
changes forced downward are struggling. Some are failing. The same will be true of 
future efforts, if more effort is not made to involve all affected parties. History is 
destined to repeat itself. 

| think the SCE process is very good. The team had to review and refine vast 
amounts of data. To be able to complete this task in such a short time is really an. 
accomplishment. | thought there should have been more guidance as to what was 
passing and what was not instead of relying on personal opinions. 

The need for local Director support is great. The need for training in many support 
areas is also great. 

My belief is that some processes should be put in place to help improve our software 
development. We have done that for the most part. However, | believe that we need 
to eliminate some “paper shuffling” to allow for more effective time management in 
performing software development. 

Much more effort needs to be focused on fixing current process problems. 
Practitioner support is vital to successful change, and many practitioners are seeing 
too little improvement for too much work. Note: this is not about misplaced focus. 
This is about too narrow a focus. Both executive/management and practitioner 
needs need to be met, not one at the expense of the other. 

| believe we'll all be better off if we can.get everyone to ask “How can ! help to make 
this happen?” instead of “Why do | have to do this? a.k.a “Not another initiative.” 
This is truly a great improvement and step in the right direction for the future. 

Too many “Chiefs” doing the work and not enough “Indians’ involvement.” 

Really need FSA director and division chief understanding. Direction and support to 
move up the maturity model. It makes all the difference. SPI takes time and 
cooperation plus special environment to take hold. 

Software shops, like SRD1, should not be required to do PCA. SRD1 does not 
produce any hardware. We should not hesitate to use the term “not applicable” when 
it is appropriate. | 

We need more emphasis on measuring our progress. If we don’t measure, we will 
never know whether SPI/CMM is working effectively and efficiently. 

Glad to see effort initiated, but need more open communication and involvement 
across the board. Worst thing we could do is let someone write procedures without 
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active involvement from those who must follow them. Standardization and 
repeatability great, but only if done so they are workable and efficient. 

SPI program managed by dates instead of ensuring processes correct. AlSs given 
deadlines that don’t consider many difficulties, risks, uniqueness. Hope FSO/FSA 
management would focus on improving processes which will improve products, but 
they continue “I'd rather have it on time” philosophy. 

Process was like blind leading blind until about Oct 96. 

Making slow progress. Seem to be going through motions, not really learning how to 
improve project. Much more cross functional involvement needed. No sense 
improving process if customer not brought in and continues to expect fire drill 
responses. 

Continue to improve - to be the best. 

| am very uncomfortable with all of this. | will do as ordered or requested to do. 

lf our customers see this system as a money-consuming, time-wasting albatross, 
how soon will it be before we’re sent packing and they're hiring an outside source to 
do the same job we could have done for them but cheaper with less demand for 
paperwork and absurd activities. 

Must happen 

Still several people who do not “buy in” to SPI program. Important for those involved 
in SPI to not criticize or bad-mouth program. More positive reinforcement might help. 
Just beginning program. Only morale problems so far. All general information does 
is make people feel they've got a lot to change. Smaller meetings geared to specific 
processes are better. Management should get everyone involved earlier in the 
process. People need to be better informed. | 

There should be more work on streamlining the SPI process. Perhaps newer tools or 
better ways to conduct meetings could be developed. Since language and software 
differ for each mission, it is hard. to have the same tools everywhere, but more 
examples might help. 

_ If lve done SPI activities, they've not been designated as such, so | don’t know when 
I’ve done them. 

It obviously is being done by some people because I've heard of it, but if I’m involved, 
or it trickles down to my level, I’m not aware of it. In other words, | do what I’m told to’ 
do, but nothing has ever been presented as “supporting SPI” 

We are getting a lot of help and usefulness out of SMS. This is due to internal 
development only, not based on help or guidance from outside our branch. 

More specific training on reviews, audits, follow-up, writing plans. 

lt appears that the push/support from the FSO is going away. I’m not sure what the 
value added is to going to Level 3, however, it is worth getting our AIS to Level 2. 
The jury's stil! out. 

Level of documentation should be reduced. Cost should be considered to ensure 
return on the investment in a relative or reasonable amount of time. 

Need more training. Need more involvement from FSO/SEPG. Need to have AlSs 
that have reached Level 2 give their documents/processes to FSO/SEPG to give to 
other AlSs trying to get to Level 2 instead of every AIS doing the same thing over and 
over and over. 

One problem exists and will continue to exist at all levels - if you develop a new 
system and use SPI from day 1, you will build a great AIS. As long as we are 
required to keep up with maintenance and new development on current AlSs to the 
tune of a more than full work week for our limited resources, SPI implementation will 
take too long. Please understand what | see - SPI is the only way to go. Upper 
management must find and develop a better way for us to get there. 

Allow systems to satisfy the CMM without using SMS. | understand most of the FSAs 
did not use the SMS for their methodology. Why is FSAIN different? 

Overall, | believe there has been a high level of success. Everything can be 
improved. This process indicates an organization’s maturity. 
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In order to get SE and middle managers really on board, they need to have a say in 
how and when. Need process not date driven. Need recognition for positive 
behavior (we still reward heroes). How many people have received 
awards/recognition for following the process or involvement in SPI. Need proper 
training. Need FSO and FSA senior management support, not just words but real 
action, money, compromise. Management needs to Jead and set the example (i.e., 
plan and track those plans). No crisis management. 

There is a subtle danger in believing a particular rating is the end of the story and 
that the hard work of improvement is behind us. A good example is the fact that few 
of the projects rated Level 2 practice configuration management with the rigor 
required to be truly effective. | 

Continued software improvement requires corporate level policy and guidance. 
Although work has been done, additional work must be completed to allow FSO 
activities to advance in maturity. 

The whole SPI effort, more than anything else, helped establish a unifying sense of 
being across the FSAs and the FSO and brought us in common business and 
engineering processes. | 

Needs to be managed through all projects with a corporate-wide vision versus 
individual projects or sites. 

This project was already utilizing the basic concepts of the SMS. The processes 
were repeatable and a relatively clean product was released to users. Utilizing the 
SMS required our group to generate a paper trail that was not previously in place, 
and hold regular formal status meetings. Communication between levels was 
improved. 

It seems to me that FSAPE should have SP! documented with all its “standards” for 
system development/maintenance and each application system should ony need - 
unique or addendum documentation requirements. 

SPI is very important and can lead to many benefits in areas of cost savings and 
customer satisfaction. It must be treated seriously and funded/staffed at an 
appropriate level to make it effective at each FSA. 
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APPENDIX C. INTERVIEW QUESTIONS 


Initial Interview Questions for FSA-KC 


Why is SPI important to FSA? 

What fucentives are there for you to upeort SPI? 

How can the SEPG help you? 

What are the anticipated hardspots in achieving CMM level 3? 

What incentives are there for you to adopt SPI initiatives? 

Are there external factors that enhance or detract com the effectiveness of SPI 


efforts? 


Interview Questions for Director, FSA 


What is the mission of FSA? 

What can software process improvement do for FSA? 

What incentives are there for key management i champion SPI? 
What is the mission of the SEPG? 


What are the anticipated hardspots in achieving CMM level 3? 


Follow-up Interview Questions for FSA-KC 


What concrete observable outcomes will result from the current process improvement 
efforts? | : 


Do the proposed changes make the employees’ job easier or harder? 
Are the changes technically familiar to members of the organization? 


Was input from the section/branch personnel included in the formulation of the 
process improvement plan? 


- What specific parts of the organization provide the most opportunity for increases in 


efficiency and quality? 


14] 





e What systems need improving, (example technological systems, management 
systems, management practices, or organizational structures)? 


e The most important thing to the leadership of FSA is.. 
¢ Whats does the director (FSA) say about SPI? What does he do? 


e What does the center director say about SPI? What does she do? 


Interview questions for Customer Representatives 
e What are your initial impressions on software process improvement? 
e Have you been briefed on SPI? 


© Will SPI initiatives help you? In what way? Are you willing to pay for SPI efforts? 
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